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SECTION 1 
INTRODUCTION 

This report presents the result of a one-year effort sponsored by the 
NASA Langley Research Center under contract NAS1-18004 to design and evaluate 
a Failure Detection and Isolation (FDI) algorithm for application to restruc- 
turable flight control. The restructurable or reconf igurable flight control 
system (RFCS) concept is a fault tolerant control concept which is capable of 
automatically generating the control action needed for recovery from unantici- 
pated emergencies as well as providing the stability and control augmentation 
for controllable flight under these circumstances. Under NASA sponsorship, 
ALPHATECH, Inc. , has been developing and testing many component technologies 
which will be necessary for near term demonstration and operational develop- 
ment of the RFCS concept. Currently, the FDI system developed for this con- 
tract is being integrated with other RFCS components for demonstration on 
NASA’s modified B-737 simulation, (see [1] - [4] for a general discussion of 
the RFCS concept). 

The FDI function is a critical component to the RFCS concept because 
there will always be conditions which can not be handled by a normal (i.e., 
any acceptable normal) control system, (which, of course, includes the pilot). 
Those failures which can not be so handled must be detected so that the RFCS 
knows when to reconf igure , and these failures must be isolated or identified so 
that the proper reconfiguration action is taken. Although there are a variety 
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of failures which may be important, those which result in a loss of control 
authority are most important since they not only result in emergency condi- 
tions, but also impact how well one can reliably respond to such conditions. 
This effort concentrated on the development of an FDI system to handle all 
such failures. These failures are known generically as control element fail- 
ures and include (though are not limited to) runaway, stuck, floating, and 
partially missing surfaces, as well as engine failures such as loss of thrust 
and stuck throttle failure modes. 

This project was divided into four major tasks. Task 1 addressed the 
issue of fundamental limits to FDI performance for the decentralized FDI 
approach previously developed by ALPHATECH, Inc. Task 2 was a preliminary 
design effort which utilized ALPHATECH' s design methodology and a preliminary 
assessment of errors. Task 3 was an algorithm refinement phase which utilized 
simulation results to provide better estimates of errors for use in the design 
methodology, and Task 4 was an assessment of critical issues for further 
development. The availability of flight recorded data from flights of NASA's 
transportation systems research vehicle (TSRV) and NASA's modified simulation 
for that aircraft motivated our application to the B— 737 aircraft. The flight 
recorded data also provided a unique opportunity for realistic evaluation of 
performance limitations in Task 1. 

1 . 1 CONTRIBUTIONS 

The detection and isolation of generic control element failures has 
received significant attention (e.g., see [5] - [13]) not only due to its 
importance for RFCS but also because of the difficulties associated with the 
need to use analytical redundancy in the solution method. Analytical 
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redundancy (unlike direct or hardware redundancy) refers to the concept of 
comparing dissimilar sensors using analytical or mathematical relationships 
between those sensors* Analytical redundancy is necessary for many control 
element failure modes because direct redundancy or built-in test equipment 
(BITE) is not available (consider a partially missing surface, for example)* 
Furthermore, those control element failures which might also be amenable to 
direct redundancy and/or BITE (e*g. , a stuck surface due to a loss of hydrau- 
lic pressure) can sometimes be handled more efficiently using analytical 
redundancy. This is because BITE, by definition, tests only the "precondi- 
tions" which are necessary for system operation (e*g*, power applied). Ana- 
lytical redundancy, however, tests the functionality of a particular system, 
thereby encompassing all modes of failures. Furthermore, it does this without 
hardware duplication, thereby reducing initial cost and weight and increasing 
overall system reliability and maintainability (fewer pieces of hardware to 
fail) as well. 

Unfortunately, the design of FDI systems using analytical redundancy is 
difficult because of the sometimes significant inaccuracies associated with 
the mathematical models which are employed. This fact provided the motivation 
for ALPHATECH 1 s development of an FDI design methodology which addresses the 
impact of such errors. One major contribution of this effort was the refine- 
ment and application of this design methodology for the control element FDI 
problem. This methodology consists of methods for choosing the structure of 
an FDI algorithm, optimizing its parameters in the presence of unavoidable 
modeling errors, and performing sensitivity analyses. It is largely based on 
the notion of discrimination metrics which can be used to bound, on an average 
basis, the decision errors associated with an FDI process. Such analyses 
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require a statistical model of system behavior which includes a wide variety 
of modeling errors (including parametric errors, unmodeled dynamics, nonline- 
arities, etc.). A qualitative analysis and several "error budgets" are used 
to derive these descriptions. 

The other major contribution is the development and demonstration of an 
FDI system for detecting and isolating all important control element failure 
modes. This system is an advance over systems which only handle a limited 
class of failure modes (e.g., [51]). The key element in developing such a 
system is the recognition that the failures which are important are those 
which result in "large" failure signatures (as later defined in this report) 
but that different failure modes give rise to different temporal signature 
characteristics which are unknown a priori . The FDI system developed for this 
project uses only qualitative information about failure signatures (we assume 
they are coherent, although this is not a requirement for the design) and 
detects and isolates failures using only failure "size" information. 

The general FDI concept used in this project is known as a decentralized 
approach because of our attempt to assess system redundancy and utilize only 
the most well-known parts of the system for specific FDI tasks. This is in 
contrast to centralized methods which integrate all information in an ' opti- 
mal" manner. Such methods work well in ideal circumstances but frequently run 
into trouble when model error exists. The loss of optimality under ideal cir- 
cumstances, which results from using the decentralized approach, is more than 
compensated by the increase in robustness to modeling errors. The first 
decentralization employed in this project is the decomposition of the control 
element FDI problem into two separate subproblems; actuator-path FDI and 
aircraft-path FDI. The actuator path problem is a "local" FDI problem which 
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is concerned only with failures which occur between the location of (total) 
actuator command measurements (e.g. , in the flight control computer) and the 
location of actuator output measurements (e.g., on a control rod). The air- 
craft path problem is concerned with all failures which occur outboard of the 
actuator output measurement. Two decoupled subsystems were designed to handle 
each of these subproblems (see Section 2). The actuator path subsystem con- 
sists of decoupled actuator path systems (one for each actuator) which make 
use of actuation models. It can handle any combination of sequential or 
simultaneous actuator path failures. The aircraft path subsystem utilizes 
models which relate aircraft motion to the measured control values and can 
only handle single failures. The ability to handle multiple aircraft path 
failures depends on knowledge of failure signatures which is not available 
without explicit control excitation (e.g., dither signals). Such an approach 
was not considered in this work. 

1.2 OUTLINE OF THIS REPORT 

Section 2 formulates the overall control element FDI problem in terms of 
the failure modes of interest, the goals and assumptions used for FDI design 
and the various configuration options available for handling the actuator-path 
and aircraft-path subproblems. Section 3 describes the decentralized approach 
to FDI and includes a variety of examples of how decentralized "residuals" can 
be generated and provides a decision structure which takes maximal advantage 
of these residuals. A design methodology for this structure is presented and 
many examples of hypothesis test designs which will be useful for the control 
element FDI problem are given. Section 4 presents a method for evaluating 
fundamental limits to FDI performance using discrimination metrics and solves 
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SECTION 2 

PROBLEM FORMULATION 

The broad goal of this project is to develop a system for detecting the 
occurrence of any control element failure (to be defined) and isolating or 
identifying the affected control. Major failures should be detected quickly 
so that appropriate reconfiguration can take place, but false alarms must also 
be minimized. 

We will assume single flight condition operation, full measurement of the 
rigid body state vector (e.g. , measurements of body referenced angular rates 
and relative wind, as well as body referenced accelerometer measurements), and 
models (not necessarily linear models) of the aircraft and actuator. Sensor 
errors including noise, scale factor and bias (within some design specifica- 
tion) and model errors including parameter errors and unmodeled dynamics (also 
within some design spec.) must be accounted for in the design process. 

The motion of the aircraft in response to control action under any opera- 
tional status (failed or unfailed) is now assumed to be representable by a set 
of differential equations, viz. 

x « f(x, 6g) (2-la) 

y = h(x, 6g) (2-lb) 

where x is some n-dimensional state vector that includes the effects of flexi- 
ble modes and disturbances due to turbulence, in addition to the rigid body 
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states which describe body referenced motion of the aircraft through the 
atmosphere; is a vector of "effective" control values; and y is the vector 
of measurable quantities. Equation 2-1 is true independent of the aircraft's 
operational status. 

Control element failures are now construed to mean: does not 

"follow" the desired control commands. That is, each of the, m, effective 
controls, Se 1 * is derived from an independent set of differential equations 
which do depend on the status of the aircraft (failed or not failed). These 
equations written in operator form are 

dg 1 = + (2-2) 

$A X “ gA 1 t 5 c 1 } + d A X (2-3) 

where 6 C represents the control commands being input to the actuation mechan- 
ism, 6 a represents the output of the actuation mechanism, gA 1 and gE 1 are 
causal operators, and dE and dA are time varying "disturbance functions, 
under no-failure conditions Eqs. 2-2 and 2-3 reduce to, 

Se* = Sa* (2-4) 

<5a X = gO A 1 t^c 1 } (2-5) 

. * 
for all i, where go a 1 is a model of a working actuation mechanism. 


*The definition of each actuator’s input and output ( 5 C and 5^) is not unique. 
For control surfaces , inputs include DFCS outputs, electrical signals between 
a DFCS and an actuator, and differential pressure in a hydraulic actuator. 
Outputs could be taken at the actuator output, on a control rod, or at the 
surface hinge. For this project we have assumed that measurements of 6^ a ud 
5 C are available. Therefore, we can define each actuator’s input and output 
by the location of the corresponding measurements. 
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Equations 2-2 and 2-3 are sufficient for describing all types of control 
element failures# These failures are decomposed into two categories. 
"Actuator-path" failures are those failures in which Eq. 2-3 differs from Eq. 
2-3 and "aircraft-path” failures are those in which Eq# 2-2 is different from 
Eq. 2-4# Tables 2-1 and 2-2 describe a variety of control element failure 
models in terms of Eqs. 2-2 and 2-3. Failure mechanisms which result in 
behavior characterized by each of the models in Table 2-1 can be conceived. 

In general, models and mechanisms depend on the exact locations of actuator 
input and output measurements. 

TABLE 2-1. ACTUATOR PATH FAILURES 


Stuck gA^S) = 0 

Floating g^(<$) = 0 

Runaway g^(<5) = 0 

Reduced Bandwidth g^(6) = SAF^) 


dA = Constant 
d A = Follows local a 
d^ = Slews to limit 
d A = 0 


TABLE 2-2. AIRCRAFT PATH FAILURES 


Stuck 


g E ( 6) = 0 

dj? = Constant 

Floating 


gE(<$) = 0 

dg = Follows local a 

Runaway 


g£(5) = 0 

dg = Slews to limit 

Partial Loss 

(A) 

g E ( <$) = k-6 

d E = 0 


(B) 

g E ( <$) = k«5 

d£ = Follows local a 


(k = 

1 - f raction-lossed) 
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The availability of actuation input and output measurements has an impact 
on the configuration of decentralized FDI systems. A centralized system would 
simply combine the actuation models with the aircraft state equation in Eq. 

2-1 and add the output measurements to the observation vector. In the decen- 
tralized approach, an assessment of the "redundancy" available from each part 
of the model is made and possible system decompositions (which utilize only 
subsets of models) are explored. Actuation output measurements allow consid- 
eration of separate actuator and aircraft-path subsystems as described below. 

Figure 2-1 describes the information flow which is available for FDI for 
some measurement configuration. Several parallel actuator paths are shown. 
Failures in each actuator can be independently detected through the use of the 
analytical redundancy which is embedded in the independent actuator models . 
That is, actuator-path failures can be detected by comparing a predicted actu- 
ator output (based on the measured input and an actuator model) with the mea- 
sured output. 



ACTUATOR • PATHS AIRCRAFT - PATH 


Fig. 2—1. Measurement Configuration and Analytic Redundancy Implications 
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When an effective control value (i.e., the control value which actually 
moves the airplane) differs from the measured output of the actuator, then an 
aircraft-path failure exists* These failures can be detected by the use of 
the analytical redundancy which is embedded in an aircraft model * That is, 
aircraft-path failures can be detected by comparing the measured motion varia- 
bles (which are a function of the aircraft states) with a prediction of these 
variables based on the control measurements* 

Clearly, from the figure, all control element failures (both actuator- 
path and aircraft-path) could be detected using an aircraft model that 
includes the actuator models thereby eliminating the need for actuator output 
measurements and reducing the cost and weight associated with the sensor hard- 
ware and redundancy management* Furthermore, such a system results in an FDI 
algorithm which is based on sensors which are more likely to survive potential 
causes of failures (e*g., battle damage, sabotage, etc.). 

On the other hand, the parallel actuator path FDI algorithms tend to be 
very simple and more reliable than the aircraft path algorithms* Also, on 
most commercial and military aircraft, the cost and weight of establishing 
actuator output measurements of some kind would not be prohibitive; (in fact, 
any servo driven actuator already has an output measurement, although not nec- 
essarily the best one in terms of failure mode coverage)* As a result, we 
expect that independent actuator path algorithms will be an important part of 
control element FDI and have, therefore, considered their development for this 
project. 

Given the development of an actuator-path FDI system, the next question 
is how to cover the remaining aircraft-path failure modes. There are two 
options in this regard. The first option is to create a backup system to the 
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actuator-path algorithm by incorporating the actuator models in the aircraft 
model and using control commands as inputs. This would then cover both 
aircraft-path and actuator-path failures. The second option is to utilize the 
control measurements for prediction of aircraft motion in the aircraft path 
algorithm. Since the latter option requires no ambiguity resolution 
(aircraft-path and actuator-path algorithms cannot disagree) and since this 
option may provide a more reliable aircraft— path algorithm (no actuator model 
errors), we have developed this second option for this project. 

Figure 2-2 shows the resulting high-level structure for the FDI algorithm 
being developed. At the top of this figure, actuator models and measurements 
are used to compare predicted and measured actuator outputs. This comparison 
consists of independent deflection residual signals. When the i th deflection 
residual is large, a failure of the i th control element is indicated. The 
decision processes for actuator path failures are decoupled and are responsi- 
ble for deciding if each residual signal is large because of a failure or 
because of model error excitation. In the lower part of the figure, measure- 
ments of various aircraft states, along with the measured deflections and an 
aircraft model, are used to form residual signals which correspond to the six 
forces and moments which define the aircraft motion (details given in Section 
5 ). The aircraft path decision process is then responsible for detecting when 
these residuals are larger than expected (accounting for noise and model 
error) and for deciding which control element failure is responsible for the 
force and moment imbalance indicated by the relative sizes of the six 
residuals . 
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SECTION 3 

A DECENTRALIZED APPROACH TO FDI 

In this section we motivate the need for a decentralized approach to the 
failure detection and isolation (FDI) problem. The term "decentralized" as 
applied to the FDI problem is used to indicate the uncoupled use of parts of a 
system model to develop "redundancy relationships," and the selective use of 
these relations in the FDI process. This decentralized approach was first 
used in the NASA F-8 sensor FDI project, [14] - [16], in order to allow clear 
trade-offs between model error and failure sensitivity to be made. Such 
trade-offs are the key to successful FDI design for systems which cannot be 
precisely described by the kinds of models which form the basis of the many 
"optimal" FDI methods, (e.g. , see [17]). This is because modeling error 
always exists (including parametric errors, unmodelled dynamics, and nonsta- 
tionary inputs) and because the optimal methods, which guarantee optimality 
under ideal conditions, are typically not "robust." 

The success of the F-8 work spawned many research activities into robust 
FDI methods. Willsky and his co— workers at MIT, ([18] — [21] first addressed 
the residual generation problem in terms of solving for "parity checks" (which 
can be interpreted as auto-regressive-moving-average models) which are insen- 
sitive to model errors. Pattipati and co-workers at ALPHATECH ([22], [24]) 
extended these ideas to include the trade-off between model error and detec- 
tion sensitivity and provided a unified framework for developing robust FDI 
methods. Weiss and co-workers at ALPHATECH, at the same time, developed a 
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control element FDI algorithm based on the conceptual framework of the F-8 
sensor work, [5J. This algorithm provided a generic structure for robust 
decisionmaking and, more importantly, began to address the need for robustness 
optimization and sensitivity analysis in the design of decision processes. 
These ideas have been extended during this project and are detailed in subsec- 
tion 3.3. 

3.1 OVERVIEW 

• What is failure detection and isolation (FDI)? 

Obviously, FDI deals with the problem of detecting deviations from normal 
behavior in specified components (sensors or effectors) and isolating the par- 
ticular component which has "failed." The key point in this sentence is that 
in order to detect and isolate deviations, one requires a specification or 
model of "normal behavior" and of the anomalous behavior to be detected. Fur- 
thermore, for each type of anomaly to be detected, these models must provide 
sufficient redundant information to allow one to detect each anomaly and to 
distinguish it from others. For example, in a triplex sensor system, in which 
there are three identical sensors of each type, one can perform voting by 
examining each triple to determine if its components are consistent (i.e., 
normal). If one sensor in the triple is significantly different from the 
other two, then we could conclude that it has failed. In this scheme, the 
model information used is that the three sensors measure the identical quan- 
tity, and the model of a deviation can be specified in several ways, such as 
in terms of manufacturer's instrument specifications. As a second example, 
consider a relatively simple and often— used check in which successive samples 
of the output of a particular sensor are examined to determine if there is an 
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obvious inconsistency. Here the model information used is a crude measure of 
the bandwidth of the variable being sensed. Finally, consider a simple system 
involving linear motion and in which one has a velocity sensor and an 
accelerometer. Here the kinematic model v - a provides a mechanism for 
obtaining one redundant relationship between these sensors. 

In the terminology used by Chow and Willsky [18], [20], the three 

examples just described are illustrations of direct (or hardware) redundancy, 
temporal (or self-test) redundancy, and analytic (or functional) redundancy, 
respectively. While there are clear differences among them, it is their 
similarities — in terms of being based on models and, more explicitly, on 
redundancy imbedded in those models — that we wish to stress. This permits us 
to construct a unified framework in which to examine and compose different 
approaches to failure detection and their robustness properties. 

• What does an FDI algorithm do? 

Roughly speaking, all failure detection systems can be described in terms 
of the conceptual block diagram of Fig. 3-1. This diagram has been used quite 
often (e.g., see [5], [23], [24], [18], [20]) and provides a framework for the 

design and analysis of robust FDI systems. In Fig. 3-1, there are two funda- 
mental parts of failure detection. The first of these is the generation of 
sets of signals (called residuals) whose deviation from "normal behavior" 
(typically meaning near zero without significant trends or patterns during no 
failure operation) can be used as the basis for detecting and identifying 
system failures. The second component of a failure detection system is the 
decision process consisting of information collection and decision-logic func 
tions. Here, the residuals that have been generated are processed in order to 
make FDI decisions. 


16 


ALPHATECH, INC. 


SENSOR 

OUTPUTS 



ROBUST HIGH-PERFORMANCE DECISION PROCESS 

RESIDUAL-GENERATION R - 2l0lA 

Figure 3-1. General Structure for FDI 

A number of issues arises in the design of each of these subsystems, and 
to begin our discussion, let us focus first on the residual generation process. 

3.1.1 Residual Generation 

The way in which residuals can be generated varies greatly. For example, 
in a triplex system, if yi(k), y200, and y 3 ^) denote the outputs of three 
identical sensors, then ri(k) * yj(k)-y 2 (k) and r 2(k) = y 2 (k)**y 3 (k) can be 
thought of as the residuals used in a voting system. In other FDI methods 
which have been suggested (e.g. , see surveys [17], [25]), Kalman filters may 
be used to generate the residuals. In some of these methods, such as the 
detection filter approach [21] and [26], [27] Kalman— like filters are 
designed, but with gains chosen in particular ways so as to make particular 
failures more readily apparent. The decentralized approach to FDI, which is 
the topic of this report, provides a generalization of the residual generation 
mechanism employed in the voting scheme described above. In this approach, 
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each piece of the system model is examined to determine individual relation- 
ships among the measured quantities. Residuals can then be generated from 
these individual relations. 

Now, although the use of Kalman filters for generating residuals may 
allow us to bypass the explicit identification of system redundancy, the 
absence of a specific assessment of redundancy frequently creates many diffi- 
culties when Kalman filter approaches are applied in a "top-down" manner. One 
of the reasons for these difficulties is the issue of robustness or model- 
error tolerance. By their very nature, good failure detection algorithms 
attempt to generate signals that are sensitive to system anomalies (i.e. , 
failures). Given that all residual generation mechanisms use models of the 
relationships among available (i.e., measured or commanded) signals, we imme- 
diately see that the possibility exists for these residuals to be sensitive to 
modeling errors as well as to system failures. What is needed, therefore, is 
an FDI method with selective sensitivity . That is, an FDI algorithm should be 
based on residuals which are maximally sensitive to failures and minimally 
sensitive to model errors. It is here that the Kalman filter approaches run 
into trouble. All residuals produced by a Kalman filter are produced using a 
centralized system model. By definition, therefore, they tend to mix together 
relationships that are known very well with those that are far more uncertain. 
For this reason centralized approaches that are optimal when models are well 
known become far from optimal when model uncertainty is taken into account. 

As mentioned above, the term "decentralized," as applied to the generation 
of residuals, refers to the identification and extraction of each individual 
source of system redundancy which we call a redundancy or parity relation. 
Selective sensitivity is achieved by examining each such relationship to 
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determine its robustness (or lack thereof) to various possible model errors 
and its sensitivity to specific failure modes. The most reliable set of rela- 
tions providing the desired coverage (i.e., capable of detecting and identify- 
ing a specified set of failures) can then be determined, and separate FDI 
tests can ben be designed. Since only the most reliable relationships are 
used in each test, the effects of model errors are minimized, and thus the FDI 
tests can be implemented reliably. Furthermore, since each redundancy rela- 
tion typically involves only small subsets of the set of available signals and 
of the set of model parameters, the problem of failure misclassif ication is 
minimized, and the effects of particular worst-case model error scenarios can 
be easily analyzed in great detail. Finally, since the resulting FDI system 
consists of a collection of extremely simple, low-order sub-algorithms, the 
overall system becomes easier to implement, verify, troubleshoot (either for 
logical errors or to pinpoint weaknesses identified during optimal tests 
caused by unanticipated sources of error), and modify. 

To summarize, in order to achieve selective sensitivity in an FDI algo- 
rithm, we generate residuals in a decentralized manner. Individual relation- 
ships between the measurable variables can then be considered in terms of 
their sensitivity to specific failure modes and to various sources of model 
errors. Only the "best" relationships for detecting and distinguishing indi- 
vidual failure modes or subsets of failure modes will then be used in the 
hypothesis tests which make up the second part of the FDI structure of Fig. 
3-1, the decision process. Details of the various decentralized residual gen- 
eration methods are given in subsection 3.2. 
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3.1.2 The Decision Process 

The decision process accumulates information about the system operation, 
performs a variety of hypothesis tests, and combines the results of these 
tests into a logical decision about the system status. 

The accumulation of information is necessitated by the fact that although 
the instantaneous value of a residual derived from any particular parity rela- 
tion provides one piece of information about possible failures, typically this 
one piece of information is not sufficient for accurate detection and identi- 
fication. Rather, the information contained in successive values of the 
residual must be accumulated over time in order to achieve acceptable levels 
of performance (see [20] for a discussion of the several ways in which infor- 
mation can be collected). 

The fact that information must be accumulated over time, coupled with the 
fact that the failure onset time is unknown, creates a situation which has led 
to a variety of failure decision mechanisms. The reason that so many methods 
have been proposed stems from the considerable advantage obtained from knowing 
or estimating the failure onset time as described below. 

UNKNOWN ONSET TIME 

The advantage of knowing the failure onset time is easily illustrated 
using simple measures of failure distinguishability (see Section 4). Con- 
sider, for example, the distinguishability of a constant, nonzero, bias which 
occurs at an unknown time ("jump failure") and measurements which are contami- 
nated by white Gaussian noise. One commonly used detection approach is to 
operate on a sliding window of data and declare a failure when the output of 
this operation exceeds a threshold. 


20 


ALPHATECH, INC. 


In particular, let y(k) represent the observed signal that obeys 

y(k) = n(k) for all k < kf, and 

under Hq: y(k) = n(k) for all k > k f 

under Hj : y(k) = m + q(k) for all k > kf (3-1) 

Now, if the decision process is defined by applying a maximum likelihood 
hypothesis testing technique over the sliding window, a decision statistic, 
S k (the log-likelihood ratio), is formed by, 

Sic ■ l — y(k-j ) - 7 (3-2) 

3-0 « n 2 

Since S k is Gaussian with equal variance under Hq and H}, we can easily 
compute the distinguishability metric, or signal-to-noise ratio of S k , viz. 

SNR{S k _ kf } - y[E{S k _ kf |Hi> - E (S k _ kf | H 0 } ] 2 /Var [ S k _ kf ] 

(k-kf) 

= for kf < k <kf + N (3-3) 

/T ®n 
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SNR{R k _ kf > = 4E=kf ^ 

Both SNR{S k _ kf } and Boi{R k - kf > are shown in Fig. 3-2. Notice that for 
large values of N, (window size), one would expect considerable improvements 
if R k were used instead of S k for decisionmaking since the failure (bias) is 
far more distinguishable (larger Bqi) in this case, especially for small k-kf. 



2135 ® 


Figure 3-2. Advantage of Known Onset Time 

Since, of course, the failure onset time, kf, is unknown, alternative 
methods which attempt to realize performance that approaches the level 
obtained when the failure onset time is known have been investigated. 

DECISION MECHANISMS 

Several "optimal" and suboptimal methods for dealing with this issue are 
discussed in [17], [25]. The optimal methods, which essentially view each 
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sample instant as a potential failure onset hypothesis, are computationally 
infeasible, and many of the subop timal algorithms which are based on these 
optimal methods can require extensive computational resources. 

In [14] - [16] and more recently in [5], an alternative approach which 
avoids these complexities while sacrificing little in performance has met with 
considerable success. This approach is shown in Fig. 3-3. In this approach, 
a trigger process produces quick responding alarms on the basis of short, 
highly sensitive tests with full coverage of all failure modes. These alarms 
are then used to generate somewhat longer running, independent tests for 
reliably identifying the failure mode and rejecting any false start. In order 
to minimize the decision delay following a failure, each failure mode may have 
a separate trigger test. The tests which are initiated by the trigger then 
provide the final reliable (i.e., desired level of error probability) failure 
decisions. In order to reject a false start from the trigger process, several 
t t verif ication" tests are triggered. These tests compare each failure mode 
hypothesis, (say H j ) , with the no-failure hypothesis (Hq). Although any 
trigger may initiate the verification process, only those failure modes which 
have been verified will be chosen by the decision logic. If all failure modes 
are not verified, a false-trigger is declared. In parallel with the verify 
tests, several "isolation" tests are used to compare each pair of failure 
hypotheses which are potentially ambiguous following a given set of trigger 
alarms. The results of these pair-wise decisions are then combined in the 
decision logic to produce failure decisions. 

There are several advantages to the decision structure just described. 
First, and most important, is that the trigger mechanism effectively provides 
an estimate of the failure onset time, kf, as the beginning of the trigger 
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Figure 3-3. General Decision Structure 


data window. This allows the use of tests based on the assumed failure onset 
time to be used for verification and isolation thereby achieving a high degree 
of decision reliability over very short time intervals. Tests such as the 
SPRT (a sequential decision mechanism requiring an assumed onset time) are 
typically very effective in realizing these advantages. In particular, these 
tests can easily be designed to be robust to unknown failure magnitudes in 
that failures which are "larger" than those considered minimal will be 
detected and isolated in a shorter period of time. 

The second advantage of the decision structure outlined above is that 
typically (e.g. , see [15], [5]) [6] the computational complexity of this algo- 
rithm is substantially smaller than the optimal and sub— optimal methods dis- 
cussed in [17]. This is in addition to the expected performance benefits in 
terms of FDI robustness. 
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Finally, in addition to the computational advantages of the procedure 
described above, the partitioning of the failure space into hypothesis pairs 
allows us to make use of residuals in a selective way so as to minimize the 
decision errors and delays of each test. That is, only those residuals which 
provide reliable information about the hypothesis pair being tested are selec- 
ted as inputs to that test. 

HYPOTHESIS TEST DESIGN 

The last issue we raise in regard to the design of FDI systems is the 
design of the various hypothesis tests which comprise the decision process. 

In classical hypothesis testing theory, such tests are completely defined by 
specification of the joint probability density function (pdf) of the sequence 
of residual signals. Such a characterization, however, is never completely 
possible since modeling errors exist, failure severity is unknown and since 
the inputs which excite the measurements are non-stationary and not always 
completely measurable. Thus, the classical theories can serve only as a start- 
ing point in the design procedure; defining a useful algorithm for hypothesis 
discrimination. Selection of parameters within this algorithm, however, 
requires a performance analysis which incorporates the uncertainties in the 
probabilistic description used to define the algorithm. 

This process is shown graphically in Fig. 3-4. The figure emphasizes the 
fact that two classes of models, truth and design models, are needed; and that 
a variety of analysis and synthesis tools need to be developed and used in 
this process. For example, algorithm structures (i.e. , equations for signal 
processing) are typically determined using a simplified design model and 
knowledge of various decisionmaking techniques. Of course, implicit knowledge 
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Figure 3-4. Hypothesis Test Design Process 


of the truth model may also be used to ensure that the subsequent sensitivity 
analyses will meet the desired specifications. The truth model is then used 
explicitly in choosing the parameters of the decision algorithm. In order to 
ensure both maximal performance and robustness to off-nominal conditions, it 
is important that the "truth" model be a statistical model which characterizes 
variations in system qualities as much as possible. Finally, where such char- 
acterizations are not possible, and where optimal synthesis techniques are not 
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available, sensitivities to important variations must be evaluated. Itera- 
tions between such an analysis and the choice of algorithm parameters can 
occur and, in some cases, iteration on the algorithm structures themselves may 
be necessary. 

Also shown in Fig. 3-4 is an evaluation of fundamental performance lim- 
its. These are really two kinds of limits of interest; one for problem feasi- 
bility and one for algorithm design. In the feasibility analysis, one asks 
the question How well can a decision mechanism perform under the best (but 
reasonable) circumstances? This topic is addressed in [8] and in Section 4 
and assumes, for example, that a detailed statistical design model exists and 
that this model exactly describes the system behavior. If adequate 
performance can not be ensured for this case, the situation is most likely 
hopeless. For algorithm design, however, a truth model which statistically 
characterizes all sources of error is needed to indicate when iterations on 
the algorithm design are getting close to fundamental limits. 

The advantage of the decentralized approach is that many of the analysis 
and synthesis tools required in this design process are readily available. 
Subsection 3.3 will detail these techniques and develop some new ones. 

3.2 RESIDUAL GENERATION 

As discussed in the previous section, the purpose of the residual genera- 
tion function is to translate redundant information about the system (in the 
form of models) into signals which exhibit a well-known and easily character- 
ized behavior (e.g. , near zero with no significant trends during normal opera- 
tion). These signals are formed through the relationships among measured 
variables embodied in the system model. 
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A variety of techniques for generating decentralized residuals have been 
developed (see [5], [22] - [24], [20]) and are described here. They are 
divided into 4 separate categories: memory less relations, finite-memory rela- 

tions, open-loop relations, and closed-loop relations. 

3.2.1 Memoryless Relationships 

Memoryless relationshps are relationships among measured variables which 
are valid at every time instant. They are easy to derive from static models 
and have been used in numerous applications from triplex or quadraplex sensor 
systems [6] to redundant arrays of inertial sensors [28]. 

In linear systems, memoryless relations are obtained as follows. The set 
of m measured variables, y, is related to the set of n "influence variables," 
x, by, 

y = Cx (3-5) 

where C is the m x n observation matrix. Memoryless residuals or relation- 
ships are formed by solving the equation w T C = 0 for all m-dimensional inde- 
pendent non-zero "parity check" vectors, w. The parity check vectors imply 
that under ideal conditions (no model error or noise), a relationship of the 
form w T y = 0 is valid. When a failure (e.g., sensor drift, large change in C) 
occurs, the residual, v, formed using this relationship, may deviate signifi- 
cantly from its nominal characteristics. That is, when no failure exists 

v = w T y = w^Cx = 0 (3-6) 

and when a failure occurs, x is something different. For example, in the case 
of triplex sensor systems, we have 
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c 

•[:] 

(3-7) 

= 

[1, -l, 0] 

(3-8) 

w 2 38 

[1, 0 , -1]. 

(3-9) 


Note that. other parity check vectors satisfy w T C = 0, but all are linear com- 
binations of Eqs. 3-8, 3-9 since wj, W£ is a basis for the left null space of 

C. 

Of course, the C matrix in Eq. 3-5 represents only a model of the 
redundancy relationships available in the static system under consideration. 
In order to develop residuals which are minimally sensitive to model errors 
and noise under normal operation, consider the uncertain system, 

y = Cj^x + n* (3-10) 

where the observation matrix is parametrically related to the random 
variable £ (representing uncertainty; £ takes on a finite number of values in 
the mathematical framework of [22], ri£ is a zero mean Gaussian noise process 
with covariance R^, and x is a random variable with zero mean and covariance 
Zi» From Eq. 3-10, the probability density function of y is 

p(y) = / p(y|&) p( £) d£ (3-11) 

where p(y | A) is a Gaussian density with zero mean and covariance, 

Cov (y|£) = + R £ (3-12) 
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In reference [24], several methods of generating robust residuals using this 
formulation are given. They include minimization of the variance of the 
residuals, minimizing the average entropy and several methods which make 
explicit use of failure models to guarantee sensitivity to particular failures 
as well as insensitivity to model error and noise. In minimizing the variance 
of the residuals, for example, we define a t-dimensional residual vector, v; 


v = wTy (3-13) 

j 

The function to be minimized is J=Trace[E{ vTv >] which can be written 


J = Tr{WT[ /CC £E *CT* + R £ ) pU) d*]W} 

= Tr{W T C W} (3-14) 


where "C is used to denote the term in brackets. If we constrain W so that it 
is non-zero, (e.g. , W T W = I), then it can be shown that since C is symmetric, 
the solution is to take the columns of W as the eigenvectors corresponding to 
the t-smallest eigenvalues of C. 

One can interpret this result geometrically by considering the following 
example* Let Ej^X and R£ = 0 for all % and let 


cos ( l) 
_sin (£) 


(3-15) 


where l is a uniform random variable on [0j, 02 ]• l n this case C can be com- 
puted and the eigenvector corresponding to the smallest eigenvalue computed. 
Through a change of variables, it can be shown that the "optimal residuals 
are computed by projecting y onto a space which is orthogonal to the vector. 


30 


ALPHATECH, INC 


9 = 


(01 + 02 ) 
cos 2 

Ox + 0 2 ) 

sin = 


This situation is depicted in Fig. 3-5: 


(3-16) 


y 2 



Figure 3-5. Geometrical Interpretation of Robust Residuals 

If the linear relationship of Eq. 3-5 is not available, it may still be 
possible to generate memoryless residuals. Suppose 

yi = f(y 2 > (3-17) 

models a static relationship between measured variables yi and y2* Then the 
residual, 

v = yi - f(y2) (3-18) 

can be computed at each time instant and used for FDI. 
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As an example of this kind of memoryless relation, consider the force 
balance relationship in a rigid aircraft model. The aerodynamic forces on an 
aircraft can be related to the relative velocity of the aircraft with respect 
to the air mass, V, the angular velocity of the aircraft about its C.G. (w) 
and the deflections of the control surfaces, 6, by 

F = f(V, w, 6) (3-19) 

This force, in turn, can be directly related to the output of the compensated 
(for off C.G. effects) accelerometer measurements. That is, since the 
aircraft obeys [29], 

m(V + ojxV) = _F + m JL (3-20) 

and the accelerometer readings (which measure specific force) obey, 

A m =V+wxV-£ (3-21) 

a set of three translational residuals can be defined by, 

v = m A m - f(V, oj, 6) (3-22) 

when air data, inertial data, and surface deflections are all measured. When 
the aircraft is operating normally, sensor errors and errors in the aero-model 
cause these residuals to deviate from zero. If a control derivative changes 
or a measured deflection differs from an actual (or effective) deflection, 
these residuals can deviate significantly from their behavior under normal 
conditions, and are, therefore, useful for detecting such failure modes. 
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3.2.2 Finite Memory Relationships 

A finite memory relation is one in which measured variables over a finite 
window in time are used. In the linear case, they can be derived similarly to 
memoryless relations as follows. Consider the linear system 

x(k+l) = Ax(k) + Bu(k) 

y(k) = Cx(k) + Du(k) (3 — 23) 

where 

x(k) = NS - dimensional state vector. 
y(k) = NO - dimensional output vector. 
u(k) = NC - dimensional input vector. 


A redundancy relationship is now defined as a linear combination of 

measurements and controls over a finite window of observation. Specifically, 

if we let Y T (k) = (y T (k), y T (k+l), ... y T (k+p)), and U T (k) = (u T (k), u T (k+l), 
P p 

... u T (k+p)), then redundancy relationships take the form 


v(k) 


WT 


Y p (k) 

U p (k) 


= W y T Yp(k) + W u T U p (k) 


(3-24) 


where v(k) is the t-dimensional residual vector which, under ideal circum- 
stances (no noise or modeling error) is identically zero, and W is the parity 
check matrix. Next, we can expand Yp(k) in terms of the system matrices 
(A,B,C,D, ) as (see [24]) 


Y p (k) = 


c 


1 

o 

• 

• 

• 

o 

Q 

! 

C A 


CB D 

• 

x(k) + 

• 

• 


• 

• 


• 

_C A P J 


_C A P_1 B **• CBD_ 


U p (k) 


(3-25) 
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or 


Y p (k) = M p x(k) + N p U p (k) 


(3-26) 


Thus the residual v, Eq. 3-24, can now be written as, 

v(k) = WT 

When no modeling error or noise is present, we can make v(k) identically zero 
by choosing W as an orthogonal basis for the left null-space of the matrix. 


M p N p 


'w' 

0 I 


Up(k) 



__ 


(3-27) 


_ A 


M 


P ~ 



(3-28) 


That is, we find all the vectors for which w T M p = 0 and form the parity check 
matrix using these vectors for its rows. 


Comments 

1. The minimum number of independent parity checks for any p is 
NO(p+l)-NS when N0(p+1) > NS. 

2. As discussed in [30], one need only look at values of p=0, ... , 

NS to find all of the independent parity checks. 

3. The solution for W can also be obtained by finding the vectors 
which satisfy W y T M p =0, and then solving Wy T N p + W U T = 0 where 
wT = (Wy T , W U T). 

Uncertainty and noise can be added to the system model of Eq. 3—23 as in 
the static (memoryless) case and similar results derived [24]. The window 
length, p, however, in this case is not as easily determined. 

In addition to the linear case above, finite memory relationships can 
also be obtained for general (nonlinear) system dynamics when sufficient state 
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measurements are available. Consider the nonlinear time-invariant, discrete- 
time system model, 


x k+l = f < x k» u k> 

(3-29) 

+ 

** 

II 

>> 

(3-30) 

z k = u k 

(3-31) 


Finite-memory nonlinear relationships between the measured variables (y^ and 
Zfc) are then, obviously, given by 

yk+l = f <yk» z k> (3-32) 

The residual, v = y^+1 “ f(yk» z k)» can then be generated. 

Several comments about these relationships and their associated residuals 
are pertinent at this point. First, note that although Eq. 3-32 is written in 
vector form, each component may be considered as a separate relationship and 
evaluated in terms of its usefulness for FDI. Furthermore, full state 
measurement is not necessary for generating those individual relationships in 
Eq. 3—32 which only depend on a subset of states. 

As an example of nonlinear finite memory relations, consider the 
so-called "rotational" residuals used in [5] and [14]. In order to form rota- 
tional residuals, we first write a discrete time, nonlinear, time invariant 
state space model for the aircraft; e.g., 

x(k+l) = a(x(k)) + b(u(k)) (3-33) 

where x(k) is the n-dimensional state vector and u(k) is the m-dimensional 
input vector. Moment balance and rotational dynamic relationships give rise 
to three components of Eq. 3-33 which correspond to the angular velocity 
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states, w(k). Since we have sufficient measurements of the states, x, and 
inputs, u, for these three equations, three rotational residuals can be 

defined by, 

Vq j = u>(k+l) - a (1) (x m (k) ) - b u (u a (k) ) (3-34) 

v r / 

where a u (*) and b w (*) are the components of a(*) and b(*) which correspond to 
the states, w, and where x m and u m are the required state and control measure- 
ments . 

3.2.3 Open Loop Relationships 

If we have insufficient state measurements for individual finite memory 
relationships, we may derive residuals in an open loop manner as follows. 
Consider the general non-linear system model of Eq. 3—29. An open-loop esti- 
mate of x k may be obtained from the recursive procedure, 

x k+l “ f ( x k» z k> (3-35) 

If the measurements are given by 

y k = h(x k ) + ^ 
z k = u k 

then a residual vector may be formed from 

v(k) = y k - h(x k ) (3-38) 


(3-36) 

(3-37) 



Comments : 


1. The residuals in 3-38 are only considered as decentralized 
if f and h represent a decoupled subset of a complete system 
model, (e.g. , an actuator model). 
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2. In order for Eqs. 3-35 - 3-38 to be useful, the system Eq. 

3-35 must be stable so that initial condition errors do not 
cause serious deviations of the residual from zero* 

As a simple example, consider a first order linear system representing, 
possibly, an actuator model* Let, 

5^(k) = x(k) 

x(k+l) = a x(k) + (1-a) 6 c (k) (3-39) 

where 

a 

A = estimated surface deflection 
x = estimator state variable 
6 C = commanded deflection 

The residual, v, is the difference between the measured surface 

A 

deflection, 6^, and the estimated deflection, That is, 

A 

v(k) = 5^^ “ $A( k ) (3-40) 

The linear model in Eq* 3-39 is typically chosen to match the DC gain and low 

frequency phase of the true actuator. In addition, rate and position limits 

can easily be added to the residual generation procedure by appropriate modi- 
fications of the estimate, 6^, at each stage. 

3.2.4 Closed Loop Relationships 

This case is the most general and is equivalent to the Kalman filter (or 
extended Kalman Filter) approach if no use is made of the natural system 
decoupling. In general, if the system and observation model are given by Eqs. 
3-29 and 3-36, 3-37 respectively, then a closed-loop residual can be formed 


from, 
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x k+ i = f(x k + Gvk, z k ) (3-41) 

vk “ y k “ h < x k> (3-42) 

Clearly, the open loop case Eqs. 3-35 - 3-40 is equivalent to the closed loop 
case with G = 0. Also, when the functions f(*,*) and h(*) are linear, it is 
possible to relate the set of finite memory relationships to the closed loop 
relationships as well. This can be seen in the one dimensional linear full 
state measurement case as follows* 

Let the system be described by. 


*k+l = AX k + B Uk 


(3-43) 


The one dimensional closed loop residual, VCL* i s given by 

x k = A[ X k -1 + GOCfc-i - *e-l)] + Bhk-1 
vcl^) = x k “ x k 


(3-44) 

(3-45) 


Obviously, when G=0 we get the open loop case, and when G=1 we get the finite 
memory case. Furthermore, rewriting Eq. 3-45 in terms of the state and esti- 
mate at time k-1, we can derive the following relationships; 


\> CL (k) = -AG v Cl.(k-l ) + VQL^ k ) 
v CL^ k ^ = vpf^(k) + A(l— G) vcL(k— 1) 


(3-46) 

(3-47) 


where denotes the finite memory residual (G-l) and voL^ k ^ denotes the 

open loop residual (G=0). 
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In general, closed loop residuals (G+0,1 in the above) will have failure 
effects which appear in all the residuals in a fairly complex manner. Further- 
more, the affects of modeling error on closed loop residuals are not easily 
characterized [6]. Thus, decentralization of the closed loop residual gener- 
ation process is possible only when the system has naturally decoupled modes. 

Finally, we note that the choice of Kalman gain, G, for FDI purposes, is 
not necessarily straightforward. The ideal choice of G would produce residu- 
als which are insensitive to model error and noise, and respond quickly to 
failures, with high sensitivity, in a well-defined and robustly distinguisha- 
ble manner, for each class of failure types. Although this is a tall order, 
if we take advantage of system decoupling, we may reduce some of the require- 
ments on G by ensuring that only a subset of failures appear in the residuals 
generated by Eqs. 3-41, 3-42 and only a subset of model errors affect the 
behavior of under normal circumstances. Some recent work in the area of 
robust Kalman filter design for FDI is given in [31]. 

3.2.5 Summary of Residual Generation Issues 

We have seen that decentralization of the residual generation process 
consists in defining residuals in such a way that both failures and model 
errors affect only small subsets of residuals. This will, in turn, allow us 
to make selective use of the redundancy relations in the decision process. 

Only those relations or residuals which provide significant information about 
a specified set of hypotheses need to be considered when defining algorithms 
for distinguishing those hypotheses. 

All of the residuals we have defined are, of course, based on design 
models. The complexity of the residual generation process is, therefore, 
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directly related to the complexity of these models. In some cases , it may be 
appropriate to use a combination of residual generation techniques to form a 
suitable set of individual or decentralized residuals. Whatever the process, 
however, it is important to recall that we will always use a model of reality 
and, in so doing, we require that an analysis of modeling error be made before 
any residual is used in the decision process. 

3.3 DECISION PROCESS DESIGN 

In subsection 3.1, we described a general structure for the information 
collection part of the decision process. This structure consists of a 
two-level process with three functional blocks: trigger, verify, and isolate 

(see Fig. 3-3). Each block is composed of a variety of statistical tests 
which are designed independently to make reliable decisions and make use of 
only the best relationships in each particular test. 

Figure 3-4 illustrated the basic methodology behind the design of these 
tests. In this subsection we will develop, largely by example, some of the 
most useful analysis and synthesis tools which comprise this methodology. 
First, however, a short review of concepts in statistical hypothesis testing 
is given. 

3.3.1 Review of Statistical Hypothesis Testing 

A great deal of literature has appeared throughout the years on the sub- 
ject of statistical hypothesis testing. Nevertheless, many results are based 
on the same fundamental concepts involving probability theory and Gaussian 
statistical assumptions. These concepts are now described. 
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UNITARY DECISIONMAKING 

The first concept typically appears in the statistics literature (e.g., 
see [B-stat]) and involves the problem of deciding whether or not an observed 
set of samples (measurements) can be described by some assumed underlying 
probability distribution. What is generally true, however, is that given this 
assumed distribution, one can only reliably reject this "hypothesis." Hence, 
in the statistics literature, one finds referral to forming a "null hypothe- 
sis" in which all hypotheses besides the one we really wish to accept are mod- 
eled by a single distribution. If the null hypothesis is reliably rejected, 
then its complement can be reliably accepted. 

To see why hypotheses can only be reliably rejected, consider the case 
where a random vector, y, is assumed to have a jointly normal distribution 
under the null hypothesis, H 0 . Denote this by y~N[m; Z] corresponding to the 
probability density function (pdf) 

Py(Y) = 1 -j2 ex P ” 7 (Y-m ) T £~ 1 (Y-m) (3-48) 

[2tt | l| ] 

where n is the dimension of the random vector, y, and |»| denotes the determi- 
nant function. 

We now wish to define a decision region, D, in which the condition yeD is 
very unlikely. Figure 3-6 shows one definition of D for the zero/mean one 
dimensional case (note that D is not unique). The area under the shaded por- 
tion of the curve is a. For multivariable zero-mean unimodel densities, we 
can choose D as follows: 


D = (Y: Pr[ By H 2 >U II 2 ] < a} 


(3-49) 
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Figure 3-6. One Dimensional Rejection Region 

where |*| denotes Euclidean norm (others are possible) and where a specifies 
the level of significance. When the random variable y is used to represent a 
residual vector and the norms in Eq. 3-49 are taken with respect to some 
covariance matrix, E, then the rejection criterion YeD is equivalent to the 
well known weighted— sum— of —squared— residuals (WSSR) test. 

In the FDI process, rejection tests are useful in the trigger process 
(which monitors operation). However, since no mention of alternate hypotheses 
(failures) is made, there is no guarantee that the rejection test of Eq. 3-49, 
if used as a trigger, would be sensitive to all important failures. In fact, 
it is certain that this test is not maximally sensitive to all failure modes. 
Thus, in failure detection, since alternate hypotheses are sometimes available, 
we need to consider binary, or in general, M-Ary decisionmaking. 
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M-ARY DECISIONMAKING 

The general problem here can be described as follows. Consider an 
observed signal y(k) which can be characterized in a probabilistic sense by 
two probability density functions (pdf's) each being valid under two different 
hypotheses. (Only 2 pdf's are considered here for simplicity, however, multi- 
ple hypotheses may also be considered). That is, we can define two condi 
tional pdfs, p(Y|Hq) and p(Y|Hi), where Y is any specified set of the signal 

y(k), (e.g., y={y(k); k=ko, ••• » kj}). 

The problem which must be solved is the definition of decision regions Di 
which map the observables, Y, into decisions about the system status. That 
is, we will decide that Hi is true when Y e Di, i=0, 1. 

In order to choose Di in an "optimal" manner, we must specify an optimi- 
zation problem. Two commonly mentioned problems are the Bayesian and Neyman- 
Pearson problems both of which are now described. 

Bayesian Hypothesis Testing — In this method we attempt to choose Di 
through minimization of an expected cost function associated with the four 
different decision regions shown in Fig. 3-7. The total cost C is given by 

C = l Cij*Pr (Y e Di | Hj ) (3-50) 

Where Cij is the cost of deciding that hypothesis i is true when in fact 

hypothesis j is true (i.e., Y e Rij). The optimization problem, 

min E{C} 

Di,i=0,l 
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DECIDE Hq 


DECIDE 


Hq TRUE Hj TRUE 


Y e Roo 

Y e Roi 

Y e Rio 

Y e Rn 


R-2346 


Figure 3-7. Decision Regions 
then, has the solution [32], 

P(Y|H].) Po (c 10 - c 00 ) 

D 1 = Y: P(Y|H 0 ) > Pi (C 0 i " Cu) 

D 0 = Y: Y^ D^(Y) (3-51) 

where ?i and Pq are the a-priori probabilities of each hypothesis. Notice 
that the form of the solution involves the comparison of the ratio of two 
pdf’s or ’’likelihood ratio”, to a threshold which is a function of the costs 
of various decision conditions. The form of this solution is quite general as 
it is also the form of the solution to the Neyman-Pearson problem described 
below. 

Neyman— Pearson Formulation — This method recognizes that the Bayesian 
costs are difficult to specify and formulates a problem in terms of specific 
performance traits. In particular, let Pp^ denote the probability of 
incorrectly deciding that Hj is true when Hq is, in fact, true. Also, let Pq 
denote the probability that the correct decision, , is made when is true. 

The optimization problem. 
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max Pj) 

subject to Pp^ = y 


(3-52) 


then, has the solution. 


where we have. 


Di = 

Dq = 


{ 


PCYlHi) 

Y: P(Y|H 0 ) > t 

{Y: Y * D X (Y) } 


} 


Pfa = 


?D = 


/ 

/ 


P(Y|H 0 ) dY 

PCYlHi) dY 
Dl 


(3-53) 


(3-54) 


(3-55) 


Thus the form of the solution to this problem is the same as the Bayesian 
case. The threshold, t, must be determined so that Eq. 3—54 is satisfied, and 
the resulting performance, P D , is determined from Eq. 3-55. The tradeoff 
between Pp^ and Pq is typically expressed by a graph called the receiver oper- 
ating characteristic (ROC) as shown in Fig. 3-8. At the extremes we see that: 


1. if t " ®, then Pp A = 0 and P D = 0, and 

2. if t = 0, then Pp^ = 1 and P D = 1. 

Of course, neither extreme is useful and the choice of operating point depends 
on some idea of acceptable performance. When this idea is expressed in terms 
of the Bayesian cost, the operating point is determined by the threshold 
choice of Eq. 3-51. 
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Figure 3-8. Receiver Operating Characteristic (ROC) 


Example - As an example of the above concepts, consider a situation in 
which under Hq, Yk is a white gaussian noise process while under Hi, y(k) is 
the same noise process plus a constant. That is. 


Ho= Yk = % 

H l : Yk = % + m (3-56) 


Now, if decisions are made on the basis of a fixed set of samples of y(k), say 
y = (y(k) : k=l, 2, ... , N}, the decision rule becomes 


N D i 

1 = l m 1 ! -1 (yj - m/2) > t 1 


(3-57) 
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where Z is the covariance matrix of the white noise process and t 1 = £n(t) 
since Eq. 3-57 is the natural logarithm of the decision rule of Eq. 3-53. The 
decision statistic, &, is called the log-likelihood ratio (LLR) and Eq. 3-57 
is an example of an "LLR test". We also note at this point that if Eq. 3-57 
operates over a moving window of data, y(k), as is frequently done in prac- 
tice, then the LLR decision statistic can, more generally, be viewed as the 
output of a finite impulse response filter. This "filter" interpretation is 
useful from the standpoint of complexity reduction since similar filters with 
fewer states can be used and analyzed. 

Returning to our example above, we wish to characterize the performance 
of the test in Eq. 3-57. Since Y is jointly Gaussian, the decision statistic 
l is a Gaussian random variable, and can, therefore, be characterized in terms 
of its mean, £., and variance, cf £ 2 under each hypothesis. These are given by 

Hq: Jq = -NmTrWa < 0 

0£ 2 = NmTirV (3-58) 

Hi ; £i = -£o ^ 0 

a * 2 = NmTjTim (3-59) 


The probability of false alarm Pp^ can be computed as a function of t^ using 
Eq. 3-58 and percentage points of the Gaussian distribution. If we let Q( £> 
represent the area under the zero mean unit normal function from ( 5 , »), then 
we have 


P FA - Q 



(3-60) 
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Furthermore,' Pq may be computed from 


p d - Q 



(3-61) 


Eqs. 3-60 and 3-61 completely specify the ROC. If we wish to make P FA = 1-P D 
(i.e., equal costs for both types of errors), it can be shown [32] that 
t l = \/2 (Ti + To) = 0 resulting in performance which is completely character- 
ized by the quantity d 2 = Gi ~ To) 2 /a £ 2 (Note that d 2 is precisely the 
signal-to-noise ratio used in Eq. 3-3). That is, larger values of d imply 
smaller probabilities of incorrect decisions. Furthermore, it can be shown 
[32] that, for any t 1 , P D can be determined from d and PpA» with larger values 
of d corresponding to larger values of P D . This is shown in Fig. 3-9. For 
the example we are considering here, 


d 2 = N ml E - ^m 


(3-62) 


Thus, we see that tradeoffs exist between performance (as expressed by d 2 ) and 
window length, N, failure size, m, and noise covariance, E. 

The Sequential Probability Ratio Test - The LLR tests described above are 
known as fixed sample size tests for obvious reasons. Another test which has 
been used extensively for FDI is the sequential probability ratio test (SPRT) 
[5] [33]* Rather than basing decisions on a fixed sample of data, the SPRT 
decides automatically when enough samples have been taken to make a reliable 
decision. That is, given P FA and P D , the SPRT chooses one of three decisions 
after each sample: Ho is true. Hi is true, or take another sample. These 
choices are completely defined in terms of an LLR statistic and two thresholds 

by 
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Figure 3-9* Detection Probability Versus d 

> t + ===> decide 
fck < t” ===> decide Hq 
t” < Jlk < t+ ===> take another sample 

where = £n E p( | ) / p( j H q ) 1 and = (y ( j ) • j ■ 1»2 k}. 

ideal circumstances, the test is guaranteed to terminate and is the 
test in the sense of minimizing the number of samples [33]* Also, 
choose t + and t~ by 



t - £n tg/l-a] 

then it can be shown that [34] Pp^ <a and 


(3-63) 

Under 
optimal 
if we 

(3-64) 
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DISCUSSION 

This concludes the review of basic hypothesis testing theory. The 
results presented show that in many cases of practical interest, the hypothe- 
sis tests which comprise the decision mechanism part of the FDI process con- 
sist of the formation of a decision statistic (e.g., the log likelihood ratio) 
and the comparison of that statistic with one or more thresholds. Optimal 
computations of the parameters of the mechanism for generating decision sta- 
tistics and the thresholds were given on the assumption that the underlying 
pdfs accurately described the behavior of the observed quantities. 

As we have argued in subsection 3.1, however, it is rarely possible to 
adequately characterize these pdfs in practice. Therefore, the design of 
hypothesis tests can use these classical ideas only as a means for determining 
algorithm structure. The parameters, however, must be chosen not for optimal 
performance in the nominal case, but for robust performance, i.e., maximal 
performance when averaged over all sources of modeling error. In the next 
subsection we go into greater detail on these ideas. 

3.3.2 Decision Process Design and Analysis 

As discussed in subsection 3.1, the classical techniques discussed above 
form only a basis for defining the structure of the hypothesis tests which 
comprise the trigger, verify, and isolate phases of the decision process. The 
parameters of these algorithms, however, must be selected so that the result- 
ing tests are robust to errors in the statistical characterizations which were 
used to develop them. The tools which will be developed in this section are 
based on the notion that these tests can be "robustif ied" by selecting their 
parameters to "optimize" and/or tradeoff desired performance measures with 
respect to a statistical truth model. That is, rather than using the 
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parameters specified applying classical theories to the design model, we wish 
to select parameters which optimize performance and achieve the desired 
tradeoffs when averages are taken with respect to a truth model that includes 
all error sources.* 

To be more explicit, we assume in this methodology that the truth model 
can be described by two pdfs for every hypothesis. The first pdf is the 
conditional distribution of the measured quantities conditioned on a random 
vector which represents the sources of error which are ignored in the design 
model. This is denoted by p(y0. Hi). The second pdf is a characterization 
of the model error vector, 9, p(0|%). The design model is based on the 
assumption that 0 takes on a specific value (usually zero). Thus, the 
structure of the decision algorithms are determined using p(y|0=0 o » Hi.). T He 
algorithm so determined will have a number of parameters, generically denoted 
by P. To select P, we go back to the original hypothesis testing algorithm 
(e.g. , max Pp subject to constant Pp^) and select P using the truth pdfs, 

p(y| H i) = /p(y|0> %) p(0|Hi) de (3-65) 

Although this concept handles many sources of uncertainty, there may be 
other unknown quantities which can not be adequately represented by a random 
variable with some known distribution (e.g., failure magnitude). In this case 
we must look at the sensitivities of the various performance measures to vari- 
ations in the unknown quantities. 

*A natural question arises here: Why not design everything for the truth 

model? The answer is that one would then expect good performance only for 
the truth model and little robustness to its assumptions. The truth model 
is not reality either. It is just a vehicle for obtaining a warm feeling 
about the robustness of the algorithm based on the design model. 
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EXAMPLE 1 (DETECTION) 

example deals with fixed sample size tests* It illustrates the use 
of the classical theories to define an algorithm structure and the use of a 
truth model to select parameters of that structure based on optimization and 
engineering tradeoffs. This example, while very simple, will be very useful 
in designing trigger tests for aircraft path failures. 

The design model for this example is given by Eq. 3-56. Under H 0 the 
observed vector process is white Gaussian noise, and under Hj this noise is 
contaminated by a constant vector. The decision process structure for a fixed 
sample size, N, is given in Eq. 3-57. Since we wish to select the parameters 
of this algorithm using a truth model, we rewrite 3-57 as 

N Di 

S - P c l yi > t" (3-66) 

j=l < 

Do 

Thus, the parameters which must be chosen (given N) are a projection vector, 

P, and the threshold, t". The truth model we will use to select these 
parameters is given by 

H 0 : Yk “ b + n k (3-67) 

Hi :yj c = b + nj c + m (3-68) 

where b is a random zero mean Gaussian constant vector whose covariance matrix 
is Eb» n k i- s a zero mean white noise process with covariance matrix, En> and m 
is a known constant vector. As discussed in subsection 3.3.1, the performance 
of the test in 3-66 can be completely characterized by the distinguishability 
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metric d 2 , however, we now compute d 2 using the truth model of Eqs. 3-67 and 

3-68. In particular we can compute the mean and variance of S under H Q and Hi 

_ _ 2 2 
(denoted by S 0 , Si, o 0 > an£ ^ 0 i) as 

S 0 = 0 (3-69) 

d 2 . a 2 = a 2 = P t(N 2 £ b + N E^P (3-70) 

?i - N P c m (3-71) 

This gives, 

d2 . _ SS£^y (3-72) 

pt(NE b + E n )P 

Recalling that large values of d 2 result in large probabilities of detection 
and low probabilities of false alarm, we wish to choose P to maximize the 
right hand side of Eq. 3-72. To do this, define the following quantities. 


c = N z b + E n , 

(3-73) 

Qt Q = C, 

(3-74) 

P = Q P, 

(3-75) 

m = Q -t m. 

(3-76) 

This allows us to rewrite Eq. 3-72 as 


d 2 = N(P C m) 2 / (pt F) 

(3-77) 


which is clearly just the square of the magnitude of the projection of m onto 
P. Thus d^ is maximized when P * Km (K is any real scalar) and using Eqs. 
3-73 through 3-76 we can solve for P as 
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P - K C-l m (3-78) 

Next the threshold t" is chosen. We recall that the structure of Eq. 

3-66 could be derived from the problem of maximizing Pp subject to a given 
value for Pp A . T° achieve the desired Pp A we nwst compute it as a function of 
t" using the truth model. Since this example deals with Gaussian statistics, 
this computation is straightforward. In particular, 

P FA = Q (t'7 o) (3-79) 

where a is defined in Eq. 3-70 and Q( •) is the error function discussed in 
subsection 3.3.1. The threshold t" is then chosen using percentage points of 
the Gaussian density. For example, to achieve Pp A = 10 -4 we need t" 
approximately equal to 3 o. 

Similarly, for a known value of m, we can compute P D using Eq. 3-61 and 
the truth model pdfs. When ’m' represents a failure, however, it is sometimes 
of interest to define a minimal failure in terms of a desired value of P D . 

This can also be done using Eq. 3-61 and the truth pdfs. For example, to 
achieve P D = lO -4 , we need (Sq - t”) =3 a. Furthermore, since d 2 (and, 
therefore, Pp) is monotonic in Si, we can guarantee that any failure magnitude 
which is larger than the minimal one defined in this way will achieve a Pp 
which is no smaller than the desired value. This is a very desirable trait 
for failure detection systems in which the size of a failure is infrequently 
known a priori. 

Another tradeoff which can be accomplished using the above computations 
is the choice of sample size, N. Since all the computations are a function of 
N, we could, for example, proceed as follows: 


I 
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1. Vary N 

2. Choose P to optimize d 2 

3. Set t" to achieve Pp A 

4. Compute minimal failure size for given P D . 

Notice that, in contrast to Eq. 3—62, as we let N — > <*>, d 2 approaches a 
finite value. That is, there is a fundamental limit to the reliability of 
decisions due to the presence of model error — an intuitively pleasing 
result. 


EXAMPLE 2 (ISOLATION) 

This example is a generalization of Example 1 and will have implications 
in terms of isolation test design for aircraft path failures. We consider 
fixed sample size tests for the problem of distinguishing the two hypotheses: 


Hi : y k = mi + n k 

»2 : yk " m 2 + n k 
The truth model is described by 

Hi : yk = mi + n k + b 
^2 : yk = m 2 + n k + b 


(3-80) 

(3-81) 


(3-82) 

(3-83) 


where mi and m 2 are two constant vectors and n k and b are the white noise and 

random bias vectors of Example 1. The LLR test obtained using the design 

model Eqs. 3-80 and 3-81 is 

N , i _ ®1 

I ( m l- ra 2) T5: n 1 yj - '2 m 1 ^ lm l + "2 m 2 S n lm 2 K c (3-84) 

3=1 °2 
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As in Example 1 Eq. 3-84 can be rewritten as, 

N °1 

S = Pt l yj l t* (3-85) 

j=l D 2 

The distinguishability metric, d 2 , calculated using the Gaussian statistics of 
the truth model in Eqs. 3-82 and 3-83 is 

d 2 = (N P t Am) 2 /o 2 (3-86) 

where o 2 is defined in Eq. 3-70 and Am = (m^ - m 2 ). Given mj and m 2 , the 
choice of P which maximizes d 2 is 

P = C — 1 Am (3-87) 

The choice of t' can be made for either Hj or H 2 in a manner similar to Exam- 
ple 1. Alternatively, a Bayesian cost can be defined as in Eq. 3—50 and the 
decision regions computed by solving the Baysian optimization problem using 
the truth model statistics defined by Eqs. 3-82 and 3-83. 

The similarity to Example 1 breaks down when we try to define minimal 
failures. This is because, for the test described by Eq. 3-85 with the pro- 
jection vector defined by Eq. 3—87, the d 2 metric (Eq. 3—86) is not necessar 
ily monotonic with failure size. That is, if the d 2 metric is computed using 
Cjmi in place of mi in Eq. 3-86, then as Ci increases from 1, d 2 may go up or 
down. In fact, it can be shown in some cases that the probability of making 
an incorrect decision may actually approach 1 as c^ approaches infinity. This 
fact can easily be seen in Fig. 3—10. The figure shows the decision region in 

the space spanned by the decision statistics lyi(k) for i = 1 , 2 (the compo- 

k 

nents of a two dimensional measurement y(k). These decision regions are 
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Figure 3-10. Failure Geometry in Measurement Space 

determined by Eq. 3-85 with a threshold that weights the costs of incorrect 
decisions (for both c^ “ 1) equally in a Baysian problem. It is easily seen 
from the figure that if the magnitude of m^ is actually larger than its design 
value, then the likelihood that the statistic lies in D 2 can become quite 
large. Thus, this particular choice of parameters is not robust to changes in 
the magnitudes of the vectors m^. 

EXAMPLE 3 (ROBUST ISOLATION) 

The problem in the previous example illustrates that we can not always 
use the structure defined by a design model and expect adequate performance. 
This is because some unknown parameters, such as failure size, result in vari- 
ations in performance which are too large to be acceptable. When this is the 
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case, there are typically two approaches which can be taken. The first and. 
most often used approach involves estimating the unknown parameters and form- 
ing a generalized likelihood ratio test [32]. While this approach performs 
well in some cases, it is also possible that, due to mismatch between the 
truth model and the real world, estimation errors can cause severe performance 
degradation. Furthermore, the analysis and design of such algorithms is com- 
plicated by the fact that a more detailed truth model is needed. The alter- 
nate approach, which is described here by example, is sometimes referred to as 
invariance. In this approach we try to develop tests whose performance is 
invariant to changes in these "nuisance" parameters. 

In this case we consider the design model hypotheses, 

% '• yk = c i m i + n k i = 2 (3-88) 

where c^ is an unknown scalar. When both c^ were known, the binary decision 
problem of Example 2 completely specified both test structure and thresholds, 
and performance measures could be computed. In Example 1, however, notice 
that one option we had in the design process was to turn the problem into one 
of rejecting H 0 with high confidence (satisfy and then evaluate what 

alternate hypothesis parameters (failure size) would ensure adequate 
performance (P D ). We take the same approach in this example for each 
hypothesis described by Eq. 3-88. We will see that Ci invariant decision 
regions using this approach can easily be described. 

We start with the problem of rejecting Hi using an N-window of observa- 
tions. Using the results of subsection 3.3.1, we can define for any value of 

ci, an a significance rejection contour by 

N N 

D(ci) = {Y k : Pr [ II £ (y(k) - ci mi) II 2 > » l (*k “ ciin)# 2 ^ a} (3-89) 

k=l k=l 
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Note y(k) denotes the random variable and Y k are sample values. The sums in 
Eq. 3-89 are taken over the N-window of data and the norms are taken with 
respect to the white noise covariance matrix to be consistent with the design 
model Eq. 3-88. Since c^ is unknown, however, we want a decision region for 
rejecting Hi such that the significance of the resulting test is invariant 
under changes in c^. That is, we want a decision region which is the locus of 
points in measurement space defined by Eq. 3-89 for all ci, or 

D = D( C1 ) (3-90) 

For two dimensional measurements, y k , this region is easily derived from Fig. 
3-11. To define this region in general, let W represent a transformation 
matrix such that z k = W y(k) (note we can consider each time instant sepa- 
rately in this case because the mean vector mi is constant). If W mi = 0, 
then p(zfc | Hi) - p(W n k | H}) which is not a function of c\. If the measure- 
ment vector is n-dimensional, then the transformation matrix W spans an n-1 
dimensional subspace. The generic rejection test would then be defined by the 
region D which can be rewritten, 

D = { Y k : Z k = W Y k and Pr[ z^C^zk) > l Z k )] < a } (3-91) 

k k 

where C z is some positive symetric matrix (usually the covariance matrix of 
Z k , though others may be desirable). 

Given a choice for C z and Ac, the region D completely defines a rejection 
test for Hi which is insensitive to changes in ci* In this example, however, 
there is an alternate hypothesis which we have not yet discussed. Since there 
is only one alternate hypothesis, H2, in this example, we would like to define 
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N 



Figure 3-11. Two Dimensional Visualization of Decision 
Region D = Complement of D 

the Hi rejection region to be maximally sensitive to H 2 (rather than being 
generic). That is, we want to define D so that the smallest possible values 
of C2 will make the probability that lies in D large. To do this, let us 
first define the rejection test based on the design model as: 

Sl = I Y k > Di ti (3-92) 

Note that this test is not in the same form as Eq. 3-91 indicates, but is con- 

sistent with the test structures in Example 2. Again, this test assumes the 
failure sign is known. To be a rejection test for Hi we know, from the above, 

that Pi must be orthogonal to mi. The remaining degrees of freedom are then 
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chosen to optimize the sensitivity to H 2 using, for example, the truth model 
of Eqs. 3-82 and 3-83. The d 2 metric is used now as a measure of sensitivity 
because, for the test defined by Eq. 3-92 with P^mi = 0, d 2 is monotonic c 2 . 
That is, 

d x 2 = [ E {S ! | H 2 } - E{Sj | Hi }] 2 / Var [Sj | H x or H 2 ] 

= c 2 2 [P 1 tm 2 ] 2 / Pit[N 2 Z b + NEjiJPi (3-93) 

Equation 3-93 is maximized for any value of c 2 by choosing Pi as follows. Let 
E = N 2 Z b + NZ n ” Q C Q» "*1 * Q p l» and “i = Q" tm i for 1 “ 2. Then the desired 

optimization problem is, 

max [ Pi c m 2 ]/Pi t Pi 

s.t. mi = 0 (3—94) 

The objective function in Eq. 3-94 is easily seen as the projection of m 2 onto 
Pi and the constraint requires that Pi lie in the null space of mi. Thus the 
solution is just the projection of m 2 onto the null space of mi* Finally 
solving for Pi, we have, 

= K Z — 1 [m 2 - (m 2 t Z - )/(mi c Z - 1 mi)mi ] (3-95) 

where K is just a normalization constant. 

We can now follow the same procedure for H 2 and obtain a test based on a 
decision statistic S 2 , which rejects H 2 with maximal sensitivity to Hi. The 
two hypothesis tests define decision regions in which it is possible to 
reliably reject each hypothesis. This is shown in the two dimensional case in 
Fig. 3-12. Note that in contrast to Example 2, there are four regions of 
interest : 
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ryk) 



Region 1) 
Region 2) 
Region 3) 
Region 4) 


can be rejected, but H 2 can't 
H 2 can be rejected, but Hi can't 
Both Hj and H 2 can be rejected 
Neither Hi nor H 2 can be rejected 


> Decide H 2 

> Decide Hi 

> Reject Hi and H 2 

> Can't decide 


Thus, these two tests allow us more flexibility in the decisionmaking process 
when events occur that can be described by neither of the hypotheses. 

Finally, we note that it would be possible to simplify this two— test 
decision process by considering the test. 


(Si - S 2 ) > ti' — > 

(Si - S 2 ) < -t 2 * > 

Otherwise > 


Decide H2 

Decide Hi 
Can't decide 


(3-96) 
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This test is not equivalent to the two test procedure. The can't decide mode 
may incorporate parts of both regions 3 and 4 of the two-test procedure depend 
ing on the choice of t]_' and t 2 '. Clearly t ^ * = t 2 ' = 0 is not a reasonable 
choice. However, the choice of tj and t2 and the relationship of these thesh- 
olds to the two-test procedure thresholds has not been fully investigated. A 
reasonable choice would be to choose tj = t 2 such that, independent of the 
hypotheses failure size (c^), the probability of making a wrong decision is 
guaranteed to be low. For example, if we want Pr[(Si - S 2 ) > t | H^) < a 
independent of cj, we assume cj = 0 (the worst case) and set the threshold at 
the a significance level for the assumed Gaussian distribution of (S^ - S 2 ). 

EXAMPLE 4 (UNKNOWN FAILURE SIGNS) 

In the above examples, we assumed that even if the "failure" magnitudes 
were unknown (c* in Example 3), the sign of the failure was known. This exam- 
ple justifies a rather obvious modification to the tests described in the pre- 
vious examples to account for uncertain signs. The modification results in 
taking the absolute value of the decision statistics, i.e., performing two- 
sided tests. 

Consider the design model of Example 1 with uncertain failure sign given 

by. 


H 0 : yk = n k 

Hi 5 yk = im + nk (3-97) 

where i takes on a value of 1 or -1. The generalized likelihood ratio test 
for an N-window of measurements is defined by. 
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Dl 

Max p(Y N | H 1# i) /Max p(Y N | H 0 , i) >< t (3-98) 

i i 0 o 


The denominator of Eq. 3-98 is not a function of i and the numerator can be 
maximized by maximizing the natural log of the numerator (assume Gaussian 
statistics and white as in Example 1). Ignoring the terms which don t 
depend on the choice of i, we get 

N 

i = arg max i[m J yjJ (3-99) 

k=l 

N 

If m and J y k are the same sign, i = 1; and if they are opposite signs, then 
k=l N 

i = -1. Thus, i = sgn (m £ y k ). Putting this solution back into Eq. 3-98, 

k=l 

taking logarithms, incorporating all the constants into a threshold and 
expressing the decision statistic in general form (as we did in Example 1) we 
have the test, 

N Di 

| P C I Yk I >< f (3-100) 

k=l D 0 


SEQUENTIAL TEST DESIGN 

The design of sequential tests such as the SPRT mentioned in subsection 
3.3.1 is somewhat different than the fixed sample size tests described in the 
above examples. Sequential tests address the binary decision problem only. 
Samples (measurements) are taken until a decision can be made in favor of one 
hypothesis or the other. The number of samples needed to complete the test is 
a random variable. Characteristics of both hypotheses must be specified in 
some detail which, for the FDI problem implies that we must choose "minimal" 
failure magnitudes and ensure that larger failures result in shorter test times. 
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As discussed at the very beginning of this subsection, we want to specify 
sequential tests using a design model and then choose their parameters based 
on a truth model. Consider as an example, the design and truth models of 
Example 1. The SPRT is easily derived and written in a general form as 

k k 

S k = Gi l (P t yj ) " l G D > t + — > Decide H X 
j=l 

< t - — > Decide H c 

< t + and > t" — > Take another sample (3-101) 

where Gj, Gp, P, t + , and t“ are parameters to be specified. Furthermore, 
since we know that model error exists, the theoretical guarantee that this 
procedure terminates is no longer valid. Thus, it is often useful to specify 
a time limit. When the time limit is reached and no decision has been made, 
we may either exit with no-decision as a conclusion or perform a fixed sample 
size test at this time. A prototype design procedure which makes use of the 
truth model is given below. 

Step 1: 

Choose a maximum sample length, N. 


Step 2: 

Select P to maximize the distinguishability metric, d 2 (N), for the 
maximal length test as in Example 1 (the answer is the same since d 2 is not a 
function of Gj or Gq) 
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Step 3: 

In keeping with the ideal parameter values Gj pt = mtljj and Gq = m t E n ra/2 

_ N 

we proceed as follows. Determine the value of S = E{£ pty^} which makes d 2 (N) 

_ k =1 

acceptable. Set Gj^ = S and Gq = S 2 /2. This results in a test in which values 
of £P fc yj less than S/2 tend to drive the statistic to its negative threshold. 

j 

Step 4: 

Determine thresholds t + = — t“. 

The last step is the most difficult since no closed form solution exists 
which relates the desired performance measures (e.g. , Pp^ and Pq) to the 
choice of thresholds when the truth model and the design model are not identi- 
cal. Reference [35] presents a numerical method which could be used for such 
calculations, however, the results are incorrect as detailed in [36]. The 
difficulty is easily illustrated by the following equations. 

Let Sk be the decision statistic at stage k in the SPRT defined in Eq. 
3-101 with thresholds t + and t“. Also, let S 1 = (Sj, S2, ..., Si). We can 
relate PpA and P D to t+ and b y> 

- ( 

P F A = I I PCS’- | Ho) dS 1 

*7 

L+A {S 1 : t“ < Sj < t + , j=l, ..., i— 1 and Si > t + } (3-102) 

N f 

P D = I I PCS 1 | Hi) dS 1 

“J 

L+ (3-103) 
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Also, we have probabilities of correct rejection (Pcr) an d missed detection 
(P MD ) which are given by, 


N 

p C r = I I p < si i H o> ds 1 

i=l 


L"& {S 1 : t“ < Sj < t + , j=l, ...» i-1 and Si < t - } (3-104) 


P MD - 



(3-105) 


Note that for N < <», we have Pp^ + P CR * 1 since there is a finite probability 

that no decision is made (i.e., t~ < Sj < t + , j=l, • ••> N). Also, Eqs. 3-102 ■ 

3-105 assume that no decision is made if e L + or L - . In some cases, it may 

°1 _ 

be appropriate to define the terminal decision rule Sjj t if S 1 ^ e L + or L . 
In this case Eqs. 3-102 - 3-105 would have to be modified. 

Equations 3-102 through 3-105 must now be evaluated using the truth pdfs 
of (as opposed to those assumed in defining as a function of y^). 
Unfortunately, closed form solutions for these equations are available only in 
the simplest cases. If, for example, was an uncorrelated Gaussian process 
(not likely, in view of equations like Eq. 3-101), then we could write 
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i-1 


P FA = I Pi° n , 
tA i=l m=l 


n (l-r m °) 


(3-106) 


i-1 


P MD “ i I 1 ^i° (1 r » lj 


(3-107) 


p No Decision ^ r ij ) 


(3-108) 


where Pi j A Pr {Si>t + |Hj> = Probability of Deciding H! at time i, given 

“ Hj and no decision up until time i 

q^j a pr {Si<t - | Hj } = Probability of Deciding H 0 at time i, given 
“ Hj and no decision up until time i 

ri j ^ 1-piJ - qjJ « Probability of deciding to take another 

sample at time i, given Hj and no decision 
up until time i. 

Note that p-jJ and q^J are easily computed if is Gaussian. 

While Eqs. 3-106 - 3-108 may provide approximations to the desired quanti- 
ties when Si is not uncorrelated, it is not clear how accurate the approxima- 
tion is or when it would break down. In order to get more accurate estimates 
of the desired quantities, numerical integration or simulation methods are 
needed. For the control element FDI algorithm developed in Section 5, some 
interesting heuristic methods of computing thresholds for sequential tests are 
developed. 


THRESHOLD SCHEDULING 

The computations described above are useful when errors in the pdf speci- 
fications can only be described in a statistical sense. If this statistical 
description adequately characterizes the worst errors throughout the opera- 
tional region (or envelope) of the system under consideration, we can be quite 
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confident that the resulting FDI design will meet its performance specifica- 
tions under all conditions. 

The problem in such a worst-case analysis, however, is that the errors 
encountered at the limits of this envelope may be so severe that exceptionally 
conservative designs result. That is, only failures of extreme severity would 
be large enough to appear significant over the worst-case modeling error. 

The natural alternative to a worst-case design is one in which we con- 
sider a limited envelope for our baseline design and modify the algorithm 
towards a more conservative design when large errors are anticipated. The 
mechanism which accomplishes this adjustment is known as "threshold scheduling" 
and has proven to be an important part of many FDI systems [5], [14] - [16]. 

The basic idea behind threshold scheduling can be illustrated by considering 
the problem of detecting inertial sensor failures using dual redundant sensors 
while accounting for misalignment errors. The residual, formed by comparing 
two similar sensors at each time, k, in this case is given by 

v(k) = yi(k) - y2(k) (3-109) 


where 


y^(k) = a^x(k) + ni(k) 

x(k) = variable being measured 

ai = actual scale factor (both equal to 1 with 
zero alignment error) 

rii(k) “ zero mean white Gaussian noise 
Thus, we can express the residual, under normal operation, as, 

v(k) = Aa x(k) + N(k) (3-110) 
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where Aa = ai~a2 and N(k) = ni - n2* Now > in order to achieve a false alarm 
rate of 1 in 10,000, we know that (using percentage points for the normal 

distribution) our thresholds (if we perform no information collection) must be 
±3.5 o N when either Aa=0 or x=0. The problem is that large values of x result 
in large residuals when Aa is non-zero, and the term Aax(k) may get, in the 
worse case, sufficiently large so that only enormous differences between y L 
and y2 can be detected with a threshold that guarantees P FA for worst-case 
errors. Threshold scheduling overcomes this worst-case performance by adjust- 
ing the threshold on the basis of a failure-insensitive, deterministic esti- 
mate of x(k) and a specification of the worst-case scale factor differential. 
Thus, the sensor FDI test becomes, 

a 

| v (k) | > 3 . 5 on + Aa nax x(k) ===> Failure Detected (3-111) 

The above example illustrates the need for threshold scheduling and the 
basic concepts involved (note, that no attempt is made here to elaborate on 
how a failure— insensitive estimate could be obtained). In order to apply 
these ideas to other FDI designs, we now describe some of the general 
principles involved. 

First, we note that the errors which are important for threshold schedul- 
ing are those whose statistical contribution to error in the residual depends 
on a deterministic signal. In the example above, we saw that the determinis- 

a 

tic signal x was used to schedule thresholds. Such a signal provides knowl 
edge about the potential for large errors at each point in time. 

Next, the statistical nature of the error source is described by specify- 
ing the pdf of the error as a function of this deterministic signal. Again, 
in the above example, we effectively assumed that Aa was uniformly distributed 
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on [— Aama-ir . Aamayl resulting in an error term which is uniformly distributed 

A A 

on [-Aa x(k) , Aa x(k)]. Note, we could also have assumed that Aa was 
max max 

Gaussian with a known variance, and obtained a different scheduling algorithm. 

The threshold schedule can now be determined by specifying the perform- 
ance characteristic which must be preserved throughout the entire operational 
envelope. For example, the threshold in Eq. 3—111 is the result of requiring 
that the false alarm performance be maintained. If missed detection perform- 
ance were to be maintained, we would need to characterize the pdf of the 
residual under failure conditions along with the error and compute the sched- 
uled thresholds accordingly. Note that we cannot maintain both performance 
measures simultaneously in a fixed sample test since only a single threshold 
can be modified. 

Sequential Test Scheduling - In sequential testing procedures such as the 
SPRT, two threshold must be scheduled. Since, in these tests, we are willing 
to declare that no decision can be made, it is, therefore, possible to main- 
tain both Pq and PpA» although as errors get large, the likelihood of not mak- 
ing any decision increases. As with the determination of nominal thresholds, 
the computations involved in determining threshold schedules for sequential 
tests are more complex than the fixed sample size case. However, a reasonable 
approach is to choose the threshold at each stage, k, such that the probabili- 
ties P(S k >t + |H 0 ) and PCS^t"!^) (conditions which, respectively, result in a 
false alarm and a missed detection at stage k) are equivalent to those 
achieved in the original SPRT with no error. (Note, S k is the SPRT decision 
statistic as, for example, in Eq. 3-101). Such a procedure is simple since 
each threshold only depends on an estimate of the impact of errors on S k 
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independent of other estimates at other stages, and has performed quite well 
in practice [5]. 

To see this, consider Example 1 with the SPRT defined by Eq. 3-101. Sup- 
pose that in addition to the random bias error in Eqs. 3-68 and 3-69, there is 
an additional error term E^, which is a function of a known deterministic sig- 
nal that is independent of which hypothesis is true. That is, the truth model 
is now 

H 0 yk “ hk + E k + b 

yjt = njt ■*" + b + m (3—112) 

Let t 0 denote the threshold we select assuming E k = 0, for all k, and let 
t E (k) denote the threshold at time k based on the criterion that the probabil- 
ity of false alarm at stage k, given that we've reached stage k, is the same 
for E^ = 0 and E k * 0. That is we want to ensure that Pr[Sfc > t Q | H 0 , E = 0] 
Pr[Sk > t E (k) | H 0 , E * 0]. Since S^ is Gaussian, this condition implies 

that : 

t Q _ S QO (k) = t E (k) - S 0E (k) (3-113) 

where S^k) = E{Sk j H 0 , E = 0}, and S 0 eOO = E ( s k I H o» E * 0). For this 
example, we have, 

k 

t E (k) « t 0 + I G I P t Ej (3-114) 

j=l 

Equation 3-114 defines a threshold scheduling algorithm for the SPRT of 
E q. 3-101. If the sign of E^ is unknown, we would like to use the worst case 
combinations and so the second term in Eq. 3-114 becomes 
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k 

l GxPt Sgn(E-jPt) Ej (3-115) 

j-1 

where Sgn(EjP c ) is a diagonal matrix of Is and -Is such that if [Ejp^ii > 0, 

then [Sgn(Ej pt) ] ^ = 1 and if EjPt < 0 then [Sgn(Ejp t ) ] a - -1. For imple- 

n 

mentation purposes we note that P t Sgn(EjP t )Ej * £ | PjEji | where n is the 

i«l 

dimension of P and Ej and P^ and Ej* denote the i^ element of P and Ej 
respectively. 


Scheduling in Single Input Single Output (SISO) Systems with Transfer 
Function Errors - The The problem to be formulated here is motivated by the 
problem of detecting "actuator path" failures as shown in Fig. 3-13. In this 
problem, we have nearly perfect measurements of both the input to the actuator 


and its output. 



Figure 3-13. Measurements for Actuator Path Failures 


Figure 3-14 shows the "open loop 11 residual generation process for linear 
single-input, single output (SISO) systems (see subsection 3.2 for definition 
of open-loop residuals). Although closed loop or finite memory residual proc- 
esses may be considered, we consider only the open loop case since it has been 
used successfully in cases where the SISO system is a stable, high bandwidth 
low-pass system. 
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Figure 3-14. Open-Loop Residual Generation in SISO Systems 


The residual process, v(t) in Fig. 3—14, is non— zero because the true 

A 

transfer function from u to y is different than the model we use to produce y. 
Typical model-errors are due to high frequency dynamics and nonlinearities 
which cannot be easily characterized and variations in hardware (over time and 
from implementation to implementation). The residual process v(t) is a non- 
stationary process because the input, u(t) is nonstationary. Furthermore, the 
size of the residual scales with the size of u(t). To see this, consider the 
case where both the true system and the model are linear systems. In this 
case, we have 

t 

v(t) = / e(x) u(t— t) dx (3-116) 

0 

where, 

e(x) = h(x) - h(x) 

h(x) = true-system impulse response 
h(x) = model 
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To design an FDI system which detects failures of the true system, we must 
first characterize the residual process when no failure exists. Since, in Eq. 
3-1 16, e(T) is not known, and in fact may vary over time and implementations 
of h( t ) , let us characterize it by the functional relationship e(x;9) where 0 
is an unknown random paramter. The parameter, 0, is the only random element 
in the description of v(t). Although u(t) could be characterized in some 
cases by a piecewise-stationary stochastic process, such a characterization 
would be of little use since we have a perfect measurement of u. 

The "size 11 of v(t) with no failure present can now be characterized as a 
function of the complete past history of the input; i.e., (u(x), xe [ 0 , t ] } • 

To see this, let us compute the mean square value of v(t) with respect to the 
variations in 0. That is, if we compute; 


v 2 = £e(v(t) 2 } 


where €q denotes expectation with respect to the distribution of 0, then v 2 
can serve as a measure of the possible size of the residual under no failure 
and used as a basis for FDI. 

To compute v 2 , we formally have, 




t 

2 

< 

ro 

It 

C£> 

i 

f e(x;9) u(t-x) dx 



1 

0 




- 



(3-118) 


t C 

/ j dti dx 2 f(xi,T 2 ) u(t-ti) u(t-t 2 )> (3-119) 

0 0 \ 


whe re 
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f ( ti > T2 ) = ^0 {e(ti;9) e(x2;0)} (3-120) 

Thus, we see that knowledge of u over the interval [0,t] and a statistical 
characterization of the error impulse response function is sufficient for 
computing v 2 . 

Unfortunately, the calculation in Eq. 3-119 is not an efficient one 
since, in general, f(xi,X2) is not seperable (if f(Ti,T2> = fi(xiu~2[x2) then 
Eq. 3—119 amounts to the product of two linear filters). To see this, let s 
consider a simple example (of no particular interest to the actuator FDI 
problem) . 

Suppose we let e(x;0) be given by, 

e(x;9) = 0e~ Qx (3-121) 

That is, the error transfer function in a first order low pass filter with 
unknown cut-off frequency, 9. This might correspond to a case where the true 
system is well known at high frequencies but uncertain at low frequencies. 

Note that Eq. 3-121 specifies a magnitude and phase relationship between u and 
v. Many characterizations of unmodelled dynamics presume only a magnitude 
relationship with unknown phase [37]. This characterization will be discussed 
subsequently. 

If we denote the probability density function of 0 by p(0), then; 

f ( x i , x 2 ) = f p(0) 0 2 e-9(V T 2 ) de (3-122) 

If, in addition, we assume that 0 is uniformly distributed over the interval 
[0L» 0 h 1 » then, using integration by parts 


76 



ALPHATECH, INC 


f(xi,T2> 



V 9 L 


0 2 e -e(Ti+T2) 


de 


(3-123) 


0 H- 0 L 


0 2 e -0(Ti+T2) 2 

3 e -0(*i + T2) (1 + 0 (ti+t 2 )) 

“(ti+T 2) (ti+T2) 


9 h 


9L 


(3-124) 


There is no apparent way to separate Eq. 3-124 into a product of functions, 
each involving only x^ or X2* In addition, even this example demonstrates 
the complexity of calculating v^, using Eq. 3-119. 

Part of the complexity in the above formulation is due to the significant 
amount of structure imposed on the error transfer function. As we mentioned 
above, transfer function errors are more commonly specified in terms of their 
magnitude response alone, with no knowledge available about phase. This leads 
us to search for scheduling methods which are based on the magnitude (or 
squared magnitude) of the Fourier transform of e(x), viz. E 2 (o>). We can do 
this using Parseval’s relation as follows. Define a positive function of 
frequency L(u)) such that L 2 ((o) > E 2 (w) for all m is guaranteed. Let N(u)) 
represent the Fourier transform of the residual sequence v(t) and U(w) be the 
Fourier transform of the input u(t). Then the following relations hold: 


n 2 ((o) = e 2 U) u 2 U) 

(3-125) 

n 2 U) < l 2 (oj) u 2 U) 

(3-126) 

oo oo 

/ N 2 (a>) d a) < / L 2 (oj) U 2 (gj) da) 

•oo — oo 

(3-127) 
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and using Parsevals relation, 

j v 2 (t) dt < J [ / |n(x) u(t-x) dx] 2 dt (3-128) 

—oo —CO —00 

where £(t) is the inverse Fourier transform of L(m). Equation 3-128 suggests 
a test which will reliably reject the hypothesis that a SISO system with trans- 
fer function error (i.e., error between reality and the model used to form a 
residual), is operating normally. The right hand side of Eq. 3-128 says that 
the input is filtered using any filter whose squared magnitude in the fre- 
quency domain bounds the actual frequency domain error. The output of this 
filter is then squared and integrated. This is then compared to the integral 
of the squared residual. The right hand side of Eq. 3—128 is the threshold 
and it is scheduled dynamically based on the temporal characteristics of the 
input. Of course, in a practical system, we would deal with these relation- 
ships in the discrete time domain and perform sums only over finite intervals 
to ensure that the values in Eq. 3-128 remain bounded. 

OTHER DESIGN ISSUES 

The design of fixed sample size hypothesis tests discussed in this sub 
section have all been based on a steady state analysis of the decision statis- 
tics which are used for decisionmaking. That is, only the steady state pdfs 
are used to optimize parameters and compute thresholds. In using these tests 
in the trigger process (see subsection 3.1), we clearly would like to minimize 
the number of samples required or equivalently the bandwidth of the resulting 
filter. This is because we recognize that there is a tradeoff between the 
size of failures one can detect and the speed in which this detection can take 
place. Some new ideas which are based on the notion of transient gain are 


derived here. 
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The transient gain of a filter H(oj) (or its Fourier transform h(t)) is 
defined as 


G T (h) = sup J T (h * y) / J x (y) 
y e L 2 (0 ,T) 


(3-129) 


where * denotes convolution, and where 

T 

J x (x) = 1/T / |x(t)| 2 dt (3-130) 

0 

In [38] J x (v), (where v is the residual vector of a Kalman filter), is 
used as a trigger statistic and the "size” of J X is established for various 
values of T using uniformly distributed modeling and sensor errors. 

The notion of transient gain is one which may be useful in designing 
trigger filters. To simplify the results which follow, we will deal with 
discrete time processes and finite impulse response (FIR) filters. The dis- 
crete version of Eq. 3-129 is 

G t (H) « sup S H X Y T 11 / fl Y x II (3-131) 


where H x is the impulse response matrix, viz. 


h 0 

0 

0 

... 0 


h l 

h 0 

0 

... 0 


h 2 

• 

h l 

• 

h 0 

... 0 

. • 

(3-132) 
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h T-l 


• • 

• • • ho 
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and Yt c = (y(0), y(l), y(T)). Now, in the design of trigger filters 

(choosing H) we would like to maximize the transient response to a failure 
subject to a constraint on the steady state variance of the output. That is, 
given the variance of the output of this filter, we can first choose a fixed 
threshold which achieves some desired false alarm rate. Of all the filters 
which result in the same output variance, we would like the one which responds 
as quickly as possible. Formally stated, we want to choose the (T + 1) 
impulse response coefficients which 

max G-pCh) 
subject to: 

hj c Cy jvp = o 2 (3-133) 

where Ivp is the vector of impulse response coefficients (i.e., the first col- 
umn of Hj) and Cy is the autocorrelation matrix for Y^. The solution of this 
problem requires a gradient scheme since both the objective and constraints 
are nonlinear in the decision vector h T . Such schemes would be facilitated by 
observing that, 

G T (H) = (Hx^) = cw (H T> (3-134) 

where X max denotes the maximum eigenvalue and o^x denotes the maximum 
singular value of the corresponding matrix. 

Other options which are similar to the above (some of which are easier to 

solve) are given below. 

1. Minimize the variance of the output subject to a constraint 
transient gain. 

2. Minimize the variance subject to a fixed step-response transient 
gain. That is, assume that y(j) is a constant in computing G T . 
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This is solved by taking the filter coefficients as Cy * times a 
vector of ones. This is also the solution to the maximization of the 
ratio of step response to variance of the output. 

3. Maximize the norm of the impulse response vector, h T , subject to a 
constant variance of the output. This is equivalent to maximizing 
the magnitude of the output at time T subject to a unit norm input 
and a constant output variance. The solution to this problem is to 
take hp as the eigenvector corresponding to the smallest eigenvalue 
of Cy Note that the similarity between this result and the robust 
parity check results of [24] are not coincidental since the problem 
statements are effectively equivalent. 

4. Maximize the worst case transient gain (by selecting y(j) to minimize 
Gp) subject to a constraint on steady state variance. That is, 
select hp to 


max min Gp(h) 
hp y(J) : j®°> ••♦> T 
subject to: 

hp*- Cy hp = 0^ (3-135) 

3.4 SUMMARY 

In this section we have developed a general structure for FDI systems and 
discussed many details associated with the design of each element of this 
structure. At the top level, the FDI function is decomposed into residual 
generation and decision processes. In the residual generation process, infor- 
mation about normal (unfailed) system redundancy (including temporal, direct 
as well as analytic redundancy) is used to form residual signals which are 
well behaved under no failure conditions and which deviate in easily charac- 
terized and distinguishable ways when failures occur. While Kalman filtering 
methods can be used to generate residuals without an explicit analysis of 
redundancy, the failure to assess redundancy and their associative errors can 
result in poor performance. This is because Kalman filtering is a centralized 
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approach to residual generation (using the system model as a whole) and will 
tend to mix well known relationships with poorly known relationships. The 
decentralized approach to residual generation breaks up the system model into 
individual relationships among measured variables (static and dynamic balance 
equations). These relationships can then be examined independently in terms 
of their usefulness in each part of the decision process. 

The decision process collects information contained in the residuals over 
time by noninvertibly compressing the residuals into various decision statis- 
tics and comparing these statistics to threshold (or to each other in some 
cases). Because of the need to process residuals over time, the unknown fail- 
ure onset time plays a large role in determining the structure of the decision 
process. Many decision mechanisms have been proposed for dealing with the 
uncertainty about failure onset. We have adopted the structure used in [14] 
which consists of trigger, verify, and isolate subprocesses. The trigger 
process is used as a quick response alarm to indicate the possibility of a 
failure. Its thresholds are set to ensure quick detection of important fail- 
ures. Furthermore, the sensitivity to all important failures are maximized by 
designing a separate trigger test for each failure mode rather than using the 
common practice of mixing all information together in a rejection test for 
normal operation (e.g. , WSSR) . The verify process is used to achieve the 
false alarm rate specifications by setting its thresholds to reliably reject 
false triggers. The isolation process is run in parallel to the verify proc- 
ess and performs binary hypothesis tests comparing all pairs of triggered 
failures. 

Finally, in designing the hypothesis tests which comprise the decision 
process, we defined a design procedure which involved four major steps. The 
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first step was to define an algorithm structure for each hypothesis test using 
a simplified "design model" which statistically describes the nominal behavior 
of the residuals under each hypothesis of interest. The second step was to 
select the parameters of this algorithm using averages taken with respect to a 
truth model. The truth model also statistically characterizes the residuals 
under the alternative hypotheses, however, it includes variations due to model- 
ing error that was neglected in the design model. Although it would be con- 
ceivable to design an algorithm for the truth model in the first place, the 
point of using separate models for design and analysis is to gain confidence 
in algorithm robustness. Finally, sensitivity analyses are made to see how 
performance varies with respect to uncertainties which are not easily charac- 
terized by a statistical truth model (such as failure magnitude). Iterations 
between this process and the first and second step may be necessary. 
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SECTION 4 

ANALYTIC METHODS OF EVALUATING FUNDAMENTAL LIMITS TO FDI PERFORMANCE 

In the generic FDI design methodology discussed in subsection 3.1, we 
argued that the evaluation of fundamental limits to FDI performance was neces- 
sary both to determine feasibility of performing the FDI function and in deter- 
mining sensitivities of performance to different system characteristics. In 
this section we develop an approach for performing these analyses based on the 
concept of distinguishability metrics. Such metrics were used in the design 
process as an objective function for the optimization of algorithm parameters. 
In this section we will assume that a statistical truth model of the observed 
quantities which adequately characterizes all important uncertainties is avail- 
able for every hypothesis (mode of operation). For characteristics which 
can't be described statistically (e.g. , failure size and signature), we will 
perform a parametric analysis of performance limitations. 

4.1 DISTINGUISHABILITY METRICS 

The basic idea behind our approach is to evaluate the distinguishability 
of pairs of hypotheses in terms of the smallest likelihood of making a n incor- 
rect decision based on some set of observations. This likelihood is nonzero 
because each hypothesis can only be described statistically. 

It is well known [32] that the decision rule which achieves the smallest 
error probability, P e , is the maximum a-posteriori (MAP) decision rule which 
achieves 
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P e - J min tpC Z p |Hi > , p(Z p |Hj)] dZp (4-1) 

Equation 4-1 is, however, not particularly easy to evaluate even using 
numerical integration algorithms and even when the densities are Gaussian. 

For this reason, it is useful to define measures which can be related to P e , 
but are easier to evaluate. Such measures and their application to FDI were 
the topic of references [22] - [24] and have found wide applicability in areas 
such as patten recognition [39], [40], control systems [41] — [43], communica- 
tions [44], information theory [45], [46], and statistics [47] - [48]. We now 
define some of these measures and describe some of their useful properties. 

If we view the pdf of a random variable, say 5 , as a vector in an infi- 
nite dimensional space, then the 'distance' between the two vectors (pdfs), 
Pi(5) and Pj(?), can be computed in a variety of ways. For example, the stan- 
dard Euclidean or 2-norm is defined as 

L 2 = J IPiU) ” Pj(?)l 2 (4-2) 

The Kolmogorov distance, or 1-norm, is defined as 

K = / | Pi( C - PjU)| d C (4-3) 

These two distance measures, while retaining some very nice topological 
properties [44], are either difficult to compute (in the case of K) or not 
easy to relate to P e (in the case of L 2 ). In reference [24], two measures 
which can be easily computed and related to P e were identified. They are the 
J-divergence which is defined by 

Pi(c> 

J ij(c) = J tPiU) - PjU)l to p ( c y d s < 4-4) 
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and the Bhattacharyya distance which is defined by 

V 2 

B ijU) = "to J IPiC PjU)l (4-5) 

Some general properties of J and B are given below. 

1. Jij > 0 

2. Jii = 0 

3. Bij > 0 

4. B U = 0 

Also note that neither J nor B necessarily satisfy the triangle inequality; 
i.e. , for i * j * k, 


Jij + Jjk * ^ik 

B ij + B jk * B ik 

The most important property of J and B is that they can be used to bound 
P e as defined in Eq. 4-1. The following relationships are derived in 
[44] - [46] 

1/2 - 1/2(1 - 4q iq j e“ 2B ij) 1/2 < P e < (qiqj ) 1/2 e _B ij (4-6) 

1/2 min(qi,qj) e~ J ij/ 8 < P e < (qiqj) 1/2 (Jij/4)" 1/4 (4-7) 

with the upper bound in Eq. 4—7 valid only for Gaussian densities. In Eqs. 
4-6 and 4-7, q* and qj are the a-priori probabilities associated with the 
two densities pi(?) and pj(c). 
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In the Gaussian case (which is of frequent interest) both metrics are 
relatively simple to compute. For example, if Pi(^) = Ntm^; C^} and 
Pj(c) = N{mj; Cj}, then we have 

F c i +c jl _1 

Bij = 1/2 in det {1/2 (Ci+Cj) Ci-l/2Cj“l/2 } + 1/8 (mj-mj ) T I j — J ( mi -mj ) 

(4-8) 

Jij = l/2Tr {CjCj~l + CjCj -1 + [Cj -1 + Cj -1 ] [mi-mj] [mi-mj ] T > - t (4-9) 

where t is the dimension of 5 . 

When the vector £ represents a time series, some other interesting 
results can be derived. Let 

t, - (z(l), z(2) , z(3) , ...z(N) ) (4-10) 

where z(k) is a stationary Gaussian vector time series. For a particular 
hypothesis, say H*, this time series is completely characterized by its (time 
varying) mean sequence, mj(k), and its covariance function, Cj(k), where, 

mi (k) = E { z(k) | Hi } (4-11) 

Ci(k) = E {(z(j) - mi (j)) (z(j+k) m i (j+k)) t | Hi) (4-12) 

The dis tinguishability metrics discussed above can be computed for the vector, 
from its mean vector m^, and (Toeplitz) autocovariance functionmatrix C^, 
which are functions of m± and in Eqs. 4-11 and 4-12, respectively. Thus, 
given an N-window of observations, all of the metrics described above can be 
computed. Although such analyses can be quite useful, it is also of interest 
to develop measures which are not dependent on an assumed window size. This 
desire leads to the consideration of asymptotic measures in which limits as N 
tends towards infinity are considered. 


C-9- 
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In [49], for example, asymptotic measures are defined for the case where 
the difference between the mean sequences is persistently exciting. If this 
is the case, then the (biased) sample correlation matrix of the difference 
between the mean sequences exists and is defined by, 

N 

Rij(k) = lim (N -1 ) £ [mi(j + k.) - mj ( j + k) ] [m^j) - mjCj)] 11 (4-13) 

N — >°° j =0 

Since m i (k), C^k) and Rij(k) are vector and matrix time sequences, we can 
take Fourier transforms element by element to obtain m£(u)), C£(u), and R£j(u>). 
The transformed quantities are then used to compute asymptotic distinguisha- 
bility measures. For example, the asymptotic B distance, is defined by 

= lim Bij(N) / N (4-14) 

where Bjj(N) is the B distance between hypotheses and Hj based on the 
N-window of data defined in Eq. 4-10. It is then shown that Bjj can be com- 
puted using a frequency domain integral. In the one dimensional case with 
equal covariance functions under and Hj(C^ — Cj — C) this integral reduces 

to, 

2tt 

lij = 1/4 (2 tt) _ 1 / Rij U) (T 1 (o>) dw (4-15) 

o 

Some interesting properties of Eq. 4-15 are described in [49]. For example, 
(Dirac) delta functions in Rij(m) can be easily handled. This corresponds to 
the case where the difference between the means is a sinusoid. The sinusoid 
is persistently exciting and, therefore, results in a finite limit to Eq. 4-14. 
Note, however, that since B^j is finite, the limit of B^j(N) is infinite. 
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In the FDI problem we are frequently interested in cases where signals 
are not persistently exciting and, hence, there is a finite limit to B^jCN). 

If this is the case, then the asymptotic measure in Eq. 4-14 is zero and pro- 
vides no information* We now extend the results of [49] for nonpersistent 
excitation in the one dimensional case with equal covariance functions. The 
general case is handled in a similar manner* 

First expand Eq. 4-15 (without the limits) in terms of the difference 
between the mean sequences. This gives 

2ir 00 N 

N^BijOO = K / doi C -1 (o)){ l e-jak N" 1 £ l mi (t+k) - mj (t+k) ] [mi(t) - mj(t)]} 


Now multiply both sides of Eq. 4-16 by N -1 and change summation variables 
(nj = t+k). This gives 


2tt 


Bij(N) = K / do) C " 1 (to) l 

o n^=0 


N 

£ e “j mni eJ^tm-j^nj^-mjCni)] [mj_(t) - mj(t)] 
t=0 

(4-17) 


Taking limits as N goes to infinity we recognize that the double summation is 
just the product of the Fourier transform of the difference between the means 
and its complex conjugate. Thus, we have, 

2tt 

BijW = lim Bji (N) = K / dm C" 1 (to) [m^w) - mj(ui)] 2 (4-18) 

N-**> o 


which is the desired result. This equation holds when the limits exist and 
this occurs only when the mean sequences have Fourier transforms. The latter 
requirement implies that both mean sequences be square integrable (i.e., have 
finite total energy). Thus, the limits do not exist for means which satisfy 
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the persistent excitation requirement of [49]. If we do admit delta func- 
tions, however, then the result is consistent with the original result in that 

Bij w is infinite. Following [49] and arguments similar to the above for the 
vector case (again assuming equal covariance functions), Eq. 4-18 becomes, 

2ir 

Bji w = lim Bji (N) = 1/4 (2it) -1 / doj [n^C w)-mj (w) ] c C -1 (<o) [m^w) - mj (oj)] 
N— >» o 

(4-19) 

EXAMPLE 

Let mj = 0, and m^OO be a narrow band process with finite energy cen- 
tered near uj c (and near 2x - co c due to sampling) in the direction v^. For 
example mi(k) = (Ep N (k) sin(u) c k)]v i where pn(k) is a pulse function of length 
N (N large). Then we have, 

Bij w = (£2/4) v^ C-Koic) V£ (4-20) 

4.2 EVALUATION METHODS FOR CONSTANT DIRECTION FAILURE SIGNATURES 
In this subsection we consider the problem of evaluating the 
distinguishability of hypotheses of the form. 

Hi : z(k) = Cifi(k) + n(k) (4-21) 

where z(k) is an n dimensional observation vector, n^ is a stationary Gaussian 
colored noise process, Ci is an n dimensional failure direction, and fi(k) is 
a failure signature. This problem is of interest since many FDI schemes 
attempt to make decisions based on observations such as Eq. 4-21 (e.g., yk 
representing detection filter residuals; the decentralized aircraft-path 
residuals to be defined in Section 5; or input disturbance estimates in the 
control element FDI scheme of [50]). 
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The distinguishability of two hypotheses Hi and Hj may be determined 
using the metrics in subsection 4.1 when Ci is known, the covariance function 
of n(k) is given and fi(k) is adequately characterized (either deterministi- 
cally or stochastically). However, for the FDI evaluation problem we are more 
interested in learning what characteristics of fi(k) are required to achieve 
some desired level of distinguishability. Thus the methods to be developed 
below will be seeking information about the "size" (e.g., total energy) and 
spectral content of the failure signature which is needed to achieve a given 
level of failure distinguishability. 

4.2.1 Detectability 

Let the hypothesis H 0 , characterized by = 0, convey the hypothesis 
that the system is in normal operation (i.e., no failure). The ability of any 
FDI system to detect % is then computed by examining distinguishability mea- 
sures between H 0 and Hi* Since z(k) is a time series, we can either make use 
of time domain measures by forming an N— window of measurements, or compute 
asymptotic measures in the frequency domain. Both cases are considered here. 

TIME DOMAIN METHOD 1 

In this method we assume that fj(k) = Es^(k) and that some particular 
sequence si(k) for k=l, ...N, is of interest. For example, si(k) = 1 might be 
used to evaluate how large the average value of fj.(k) needs to get to be 
detectable; or si(k) = sin (tu c k) might be used to evaluate the frequency 
regions of greatest detectability. Let £ be defined as in Eq. 4-10 with mean 
under Hi, m^*-, and covariance matrix under both Hi and H 0 , C^. If is a 
desired level of detectability (determined using the bounds of Eq. 4-6, then 
we can compute the value of E which achieves 3^ from 
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E m i„ 2 ■ ®dl®t tc { ‘“cl 1 (4-22) 

Furthermore it is easy to see that for > ^min^* ® *- s l ar S er implying 
greater detectability for "larger" failures. Note, however, that many smaller 
failures may also be frequently detected by an FDI scheme. This is because 
Eq. 4-22 merely gives the failure size such that both the probability of false 
alarm and the probability of missed detection are small. Thus, for an FDI 
test which achieves the desired Pp^* smaller failures (than Emin) coul d be 
detected, but with less reliability (i.e., larger Pi®)* 


TIME DOMAIN METHOD 2 

In this method, we also assume that fj.(k) = Es^Ck) , but ask the question 
How large must E be so that the worst case sequence sj.(k), where si(k) is 
constrained to have unit energy, is detectable? Again, using x. as defined in 
Eq. 4-10, the covariance matrix under and H 0 , C ? , is easily computed. The 
mean vector is expressed by 


= E C^sn 

where % is an nN x N matrix defined by, 

f"Ci 0 0 ... 0 

Ci 0 ... 0 

... Ci 

and sn is an N x 1 vector defined by, 

s N t = (s i ( 1 ) , .... s i(N) ) 



(4-23) 


(4-24) 


(4-25) 
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The worst case signature with, for example unit energy, is defined as the one 
which causes B to be a minimum. Thus s N is found by solving, 

min 1/8 E 2 sjqt S N 

s • t • 55 1 (4-26) 

The solution to Eq. 4-26 is found using Rayleigh's inequality: sjj is the 

eigenvector of the matrix in brackets corresponding to its smallest eigen- 
value, A m i n > and the minimum is, therefore, E 2 A m { n . Thus, E m £ n 2 , the smallest 
value of E 2 which gives B - is just, 

Emin 2 = ® ®d / ^min (^”27) 


TIME DOMAIN METHOD 3 

The last time domain method is directed at evaluating average limits when 
the failure signature fi(k) can be described stochastically. As we will see, 
this method results in extreme conservatism in terms of detectability and is, 
therefore, not recommended. The extreme conservatism comes from the fact that 
f^(k) is described as a zero mean Gaussian process. Although this may in some 
cases be an accurate description of behavior under the "failure” hypothesis, 
the utility of labeling undetectable signals (i.e., frequently near zero) as 
failures is questionable. 

In this case, we let f*(k) - Esi(k) be described by a first order 
difference equation. 


Xf (k) = A f x f (k-1) + B f w(k-l) 

s^(k) = CfXf(k) + Df w(k) (4-28) 
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where x f is an m dimensional state vector and w(k) is a Gaussian white noise 
sequence. Using Eq. 4-28, fj(k) is a zero mean Gaussian process with autoco- 
variance function Z f (k) = E 2 Z s ( k >> where E s (k) is the autocovariance function 
of s^(k) (computed from Eq. 4-28, e.g., see [22]). Define C as in Eq. 4 10, 
with m ? = 0 under Hi and H 0 , and with autocovariances. 


Z Q (k) = E[z(j)z t (j+k) | Hq] 

Ei(k) = E[z(j)zt(j+k) | Hi] = Eo( k > + C i C i t ^ f (k) 
Plugging these statistics into Eq. 4-8 yields, 

B = 1/2 Ln det { [EiE 0 _1 ] l/2 + l Eo^i -1 ] l/ 2 > 


(4-29) 


(4-30) 


(4-31) 


where Zi and I 0 are the covariance matrices of c under % and H 0 respectively. 
Equation 4-31 can be simplified using Eq. 4-30 for more efficient calculation. 
However, its extreme conservatism makes it a poor choice for FDI evaluation. 
To see this, consider the case where N = 1, Zf = 1 and z(k) is a one dimen- 
sional process. For the high signal to noise ratio case (Ci 2 » Z Q ), Eq. 4-31 
is approximately 


B ~ 1/2 Ln [(Ci 2 /Zq + D/2] 1 / 2 


(4-32) 


For a reasonable value of B (say 10) Eq. 4-32 implies that Ci must be on the 
order of 10* times greater than Z Q l/ 2 in order to be detectable! Furthermore 
the conservatism of Eq. 4-32 is not entirely due to the looseness of the guar 
anteed performance bounds in Eq. 4-6. In the one dimensional case the value 
of Ci which achieves any desired P e (for equal false alarm and missed detec- 
tion probabilities) can be computed. To achieve P e = 10 4 (corresponding to 
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about B = 10 for the upper bound in Eq. 4-6) we require that .01(Ci 2 + 

~ 2.5 z Q l/2 (this is derived using Eq. 4-1 knowing that the pdfs under the two 
hypotheses cross at only two points). Thus even using more exact calculations 
for the one dimensional case requires that be about 250 times as big as 
E l/2 to be detectable. Although this is considerably smaller than Eq. 4-32 
would indicate, it is still exceptionally conservative.* 

FREQUENCY DOMAIN METHOD 1 

Since z(k) is a time series, we can use the asymptotic measures described 
in subsection 4.1 to evaluate detectability. In none of these methods will 
f^(k) be described as a zero mean process due to the discussion above. How- 
ever, such methods are easily derived. In the first method, as in the first 
time domain case, we assume that some aspect of f^(k) is known. To use the 
frequency domain integral in Eq. 4-19, however, it is evident that only the 
squared Fourier transform of f^(k) (its power spectrum) needs to be specified. 
Thus, this analysis is not specific to a particular signature, but is valid 
for the class of failure signatures which have the same power spectrum. Let 
S n (m) denote the spectral density of n(k) , and fj/k) = Esi(k) with s^(k) a 

unit energy signal with power spectrum m s 2 (m) (i.e., f m s 2 (w) dm = 1). Then, 

o 

to achieve B w = B^, Eq. 4-19 requires, 

2ir 

E min 2 = B d/ C 1/4(2 tt) — 1 / m s 2 (m) Ci t S n ” 1 (w) C* dm] (4-33) 

o 


*Note that in the one dimensional nonzero-mean case, calculations using B dis- 
tance instead of actual error probabilities result in estimates of signature 
size which are about a factor of 9 too big. 


95 



ALPHATECH, INC. 

Candidates for m s (uj) include wideband processes (e.g., the Fourier transform 
of e -ak , or p N (k); a pulse of length N) and narrowband processes (e.g., the 
Fourier transform of p N (k) sin(u> c k)). For wideband processes it may be of 
interest to plot the integrand of the denominator of Eq. 4-33 vs. u to high- 
light the regions in the frequency domain which contribute most to detecta- 
bility. For the narrow band processes, plots of E min vs. u> c also indicate 
frequency. regions which are highly detectable. 

FREQUENCY DOMAIN METHOD 2 

Paralleling the development of time domain methods above, we again assume 
fi(k) * Esi(k) and find E such that the worst case signature spectrum , m s (u), 
is detectable. The worst case signature minimizes B w subject to the unit 
energy constraint on si(k). Thus we must first solve, 

2ir 

min / E 2 m s 2 (w) Ci t S n -1 (aj) C^ daj (4-34a) 

o 

2tt 

s.t. J m s 2 (m) do) = 1 (4-34b) 

o 

Since all factors in the integrand of Eq. 4-34a are positive, the solution is 
for m s (u) to be a narrow band process at the frequency for which C^ 

is minimum. The minimally detectable energy is then computed as in Eq. 4-33. 
This solution is, unfortunately, of little use in practical problems since S n 
must be estimated and, therefore, the exact frequency which results in minimal 
detectability is of little interest. The dual problem to Eq. 4-34, however, 
may be of interest. That is. 
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2u 

min E 2 f m s 2 (aj) dm 
o 

2ir 

s.t. 1/4 (2-rr)" 1 / E 2 m s 2 (o)) Ci t S n ~ 1 (w) do) = B d w (4-35) 

o 


The solution to Eq. 4-35 is not immediately obvious* However, a similar 
problem can be stated which has a simple solution* Let f(u>) - E 2 m g 2 (a)), and 
g(o>) « Ci t S n “ 1 (oj) Ci* The constraint in Eq. 4-35 can now be rewritten using 
inner product notation to denote the integral over the interval [0, 2ir] , as 

<f,g> = B d w (4-36) 


The minimum norm function, f, which achieves Eq* 4-36 can be easily found* It 
is, 

f = [B d w/ <g,g> ] g (4-37) 


Thus, we have, 

2tt 

f*Co>) = [C i t S n - 1 (o)) Ci ] B d w / [1/4 (27T)- 1 / (CitS n -lU) Ci)2 d(IJ ] (4-38) 

O 


as the solution to 

2tt 

min f f 2 (m) do) 
o 

2tt 

s.t. 1/4 ( 2 it)“l / f(w) Cits n -l(to) Ci dw = B d w ( 4 - 39 ) 

o 


The energy in the signature f(k) needed to achieve Eq. 4-37 is computed from 


Eq* 4-36 using 


2tt 

E min 2 = / f* (u») da, ( 4 - 40 ) 

o 
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4.2.2 Distinguishability 

Failure distinguishability refers to the inherent ability to distinguish 
failure modes from each other (not including the no-failure mode). Although 
any of the calculations discussed in subsection 4.1 provides, in principle, a 
direct means for evaluating failure distinguishability , there is a problem in 
that the signature energies for different failure modes must be specified. 

One logical approach might be to use the minimally detectable energies deter- 
mined from a detectability analysis such as those described above. However, 
it is possible to show that such an approach does not guarantee that larger 
failure signature energies are more distinguishable. In order to provide a 
more meaningful distinguishability analysis in this regard, we define a new 
distinguishability concept. 

As is well known in the statistical literature, the assumption of a par- 
ticular statistical behavior for an observation under some hypothesis, , 
allows significance only in the testing of the null hypothesis, H*. That is, 
given a model for H*, we can only design a test which reliably rejects that 
hypothesis. Now suppose we wish to define a region in measurement space which 
represents a highly significant decision that the i®"^ control element is not 
failed. Furthermore, suppose we wish to define this decision region so that 
the resulting test is invariant with respect to the failure signature energy 
Ei. 

As in subsection 3.3, let the a-signif icance level test for a known 
signal energy (consider one sample of z(k)) be defined by 

D(Ei) = {Z : Pr [Hz(k) - C^Ck) « 2 > HZ - CjfiCk) I 2 ] < a} (4-41) 
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The Ei invariant decision region is then the locus of points in measurement 
space defined by Eq. 4-40 for all possible Ej, or 

D = D D(E^ ) (4-42) 

E i 

Thus, as in subsection 3.3, the decision region, D, is defined by projecting 
the a-significance contour for any E^ into the null space of C^. 

We can now define the minimum signature energy needed to distinguish 
failure i from failure j, Ei/j , as the minimum signature energy required to 
achieve some desired value of a given metric (e.g. , B w ) after projection into 
the null space of failure j. Notice that this definition is not symmetric 
(i.e., Ei/j * Ej/i). Furthermore, note that this analysis quantifies the 
limits to performance of the "isolation" hypothesis tests discussed in subsec 
tion 3.3 
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SECTION 5 

CONTROL ELEMENT FDI DESIGN AND EVALUATION FOR THE B-737 AIRCRAFT 

In this section, the ideas described in Sections 3 and 4 are applied to 
the control element failure detection and isolation problem. Aircraft-path 
and actuator-path failures are considered in independent subsystems as dis- 
cussed in Section 2. For both subsystems we have assumed that all sensors 
have been validated (i.e., contain only "in-spec" errors). For the actuator- 
path subsystem, multiple simultaneous and sequential failures are allowed. 

For the aircraft-path subsystem, only single failures are allowed. Also, no 
aerodynamic effects other than those due to control element failures are 
allowed (note: such effects are likely to be detected, however, isolation 

performance is severely affected). 

Both subsystems make use of the "trigger/verify/(isolate) structure dis 
cussed in subsection 3.1 for handling the unknown onset- time problem. In the 
aircraft— path subsystem, we have concentrated heavily on developing a formal 
design methodology which allows the designer to assess FDI performance capa- 
bilities and limitations as a function of both sensor noise and model uncer- 
tainty. This was done to convert, as much as possible, the typical algorithm 
tuning process into one of validating model error assumptions. Except for 
defining threshold scheduling procedures, this has been successfully 


100 


ALPHATECH, INC. 

accomplished. For the actuator path subsystem, two separate hypothesis test- 
ing algorithms were examined. The first algorithm was designed using only 
static thresholds which are selected on the basis of average error budgets. 

The second algorithm used threshold schedules whxch assumed that all error 
sources could be bounded by a high-pass transfer function error. 

In the remainder of this section, we will detail the design, development, 
and evaluation of both aircraft-path and actuator-path control element FDI 
systems. Evaluation of the algorithms was accomplished using data generated 
from NASA's modified B-737 simulation. This simulation is a six degree of 
freedom simulation with nonlinear aerodynamic models, realistic actuation mod- 
els (high order dynamics, rate limits, cable stretch, etc.), a Dryden wind 
model, and sensor errors including white noise, biases, and scale factors. In 
the case of aircraft— path failures, we will also evaluate fundamental limits 
to FDI performance based on flight test data obtained from NASA's Transporta- 
tion Systems Research Vehicle (TSRV). 

5.1 AIRCRAFT PATH FAILURE DETECTION AND ISOLATION 

The aircraft path FDI problem described in Section 2 is an attempt to 
characterize a very broad class of failure modes for which a single FDI system 
is desired. The failure modes considered as aircraft path failures are those 
failures in which the effective control value (i.e. , the value of unfailed 
control "deflection" which results in the same applied aerodynamic forces and 
moments as the failure mode) is different than the measured control value. 
Thus, any failure occurring outboard of the measured control value is consid- 
ered as an aircraft path failure. This includes the traditional "partially 
missing" aircraft path failure mode as well as stuck, floating, runaway, etc.. 
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when these failures occur in the aircraft path. To achieve such generality in 
the FDI system, specific temporal characteristics of failure signatures will 
not be used explicitly. Thus, systems such as [51] are not of interest. The 
OSGLR method described in [10] provides insensitivity to failure signature 
information through estimation of temporal basis coefficients. In [52] this 
method was applied using standard Kalman filtering techniques to generate 
residuals. However, the method is also applicable to other types of residu- 
als. In our approach, we make a broad assumption that the failure signature 
is coherent. That is, we assume that the signatures of important failures 
remain large and of one sign throughout the length of time needed to detect 
and isolate failures. This assumption leads to specific structures for the 
FDI process. We then concentrated our efforts in developing analytical design 
and evaluation methodologies for selecting parameters and predicting perform- 
ance. In this section we detail the application of these methodologies. 

5.1.1 Decentralized Residual Generation 

The first step in designing a system to generate residuals for aircraft 
path FDI is the assessment of the various sources of analytic redundancy which 
are available. 

Figure 5-1 shows the interrelationships of the various elements which 
describe the dynamics of a rigid body aircraft. The vector equations and var 
iable definitions are shown in Table 5-1 and are derived in [29]. All quanti- 
ties are in c.g. — centered body-axis coordinates (unless specified otherwise). 
(In addition to these elements one might include relationshps between actuation 
effort (hinge moment) and V, 6, m, and g as a potential source of redundancy. 
However, these are typically unreliable [53]. 
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KEY: *- 2352 


FB = FORCE BALANCE 

MB = MOMENT BALANCE 

TD = TRANSLATIONAL DYNAMICS 

RD = ROTATIONAL DYNAMICS 

TK * TRANSLATIONAL KINEMATICS 

RK = ROTATIONAL KINEMATICS 


Figure 5-1. Generalized Rigid Body Aircraft Model 


TABLE 5-1. ELEMENTS OF AN AIRCRAFT MODEL 


FB: F a — f(Vy, > 6) 

MB: M a = g(V w , av, 6) 

TD: m(V + w x V) = F a + n£ 

RD: Iw + « x Iw - M a 

TK: P = Rb 1 (0) V 

RK: 0 = H(0) _1 a) 

F a = Aerodynamic force vector 

M a = Aerodynamic moment vector 
V = Translational velocity vector 
a) = Angular velocity vector 
P = Position vector in ECEF coordinates 
6 = Euler Angles 

V w = Translational velocity of aircraft 
relative to air mass H V 
- Rotational velocity of aircraft 
relative to air mass = ui 
6 = Surface deflections 
£ = Acceleration of gravity 
m = Mass 

I = Moment of inertia matrix 
Rg 1 = Rotation matrix from body-axes to 
ECEF axes 

H = Transformation matrix (see [10]) 
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Although several aircraft relationships could be considered in the resid- 
ual generation process, it is the force and moment balances which provide the 
most useful information about aircraft path failures. This is because these 
relations are the only ones affected by failures and because other relations 
have potential for adding only uncertainty due to model error (e.g. , effects 
of wind acceleration on TD). 

If measurements of both inputs and outputs of these relations were 
available, static or memoryless residuals could be generated. For the force 
balance relation, this is the case because accelerometer measurements are 
available. That is, let A m be the measured specific force from three orthog- 
onal accelerometers centered along the body axes. Then we have 

A m = V + a)xV-g (5-D 

The force balance equation in Table 5-1 then allows the formation of an 
independent estimate of these measurements from air data, angular velocity, 
and deflection measurements. A set of three translations residuals can then 

be defined by, 


v T = vy = ^ - f(V), w, S)/m (5-2) 

VZ 

For the moment balance relation, memory less residual generation would 
require direct measurement of aerodynamic moments. Since these are not avail 
able we will use the rotational dynamic relations in addition to the moment 
balance equations to form finite memory "rotational” residuals as follows. 
First, write a discrete time, nonlinear, time invariant state space model for 
the aircraft, viz.. 
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x(k+l) = a[x(k) , 6(k)] (5-3) 

where x(k) is the n-dimensional state vector and S(k) is the ra-dimensional 
input vector. The moment balance and rotational dynamic relationships give 
rise to the three components of Eq. 5-3 corresponding to the angular velocity 
states, w( k). Since we have complete measurements of the states, x, and 
inputs, 6, for these three equations, three rotational residuals can be 

defined by, , 

vp \ 

vq J = io(k+l) - a^lxCk), S(k)] (5-4) 

VR / 

where a u) (*) is the component of a( •) which corresponds to the states, to. 

The rotational and translational residuals defined above are clearly 
decentralized in that they only use those parts of the system model which con- 
tain direct information about the failures of interest. In an ideal world, 
the other relations provide the information which would be necessary to opti- 
mally incorporate all measurements into the residual generation process 
through use of a Kalman filter. However, errors in these relationships can 
make such an approach suboptimal for the failure detection problem. With the 
decentralized approach we are sure that the errors in unused relations will 
not have any effect on FDI performance and for those errors which do, analyti- 
cal evaluation of their impact will be possible. 

The use of static and finite memory residual formulations is chosen for 
the simplicity of failure characterization. In particular, we see that, for a 
broad class of force and moment relationships, failures will show up in fixed 
directions in the residual space defined by Eqs. 5-2 and 5-4. For this to be 
the case, the functions f(«) and a u (*) must be of the form. 
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f(.) = fi(x) + l bi gi(6i) 
i 

(5-5) 

a u)(‘) = a l( x ) + I d iSi< 5 i) 

(5-6) 

where, again, x is a vector of measureable states needed in Eqs. 5-2 and 5-4. 
The failure direction corresponding to the i c ^ control element failure in the 
residual space defined by v c = (up 11 , VR fc ) is then C± z = (bi C , di c ). Although 
Eqs. 5-5 and 5-6 may seem restrictive, they are in fact typical of standard 
aircraft models (i.e., linear aerodynamic models). The details of these equa- 

tions for the B-737 are now given. 

The standard non-dimensional description of forces 
to a body fixed coordinate frame takes the form 

and moments referenced 

X = O’ • S • CXB 

(5-7) 

Y = Q • S • CYB 

(5-8) 

z = Q . S • CZB 

(5-9) 

L = CJ • S ♦ b • CLB 

(5-10) 

M = O' . S ♦ c • CMB 

(5-11) 

N = "Q • S • b • CNB 

(5-12) 


where represents the dynamic pressure (in units of lbs/sq. ft.), S represents 
a reference surface area (sq. ft.), b is the reference wing span (ft.), and c 
is the reference mean chord length (ft.). The non-dimensional coefficients 
(CXB , CYB, CZB, CLB, CMB, CNB) at any point in time are typically related to 
the instantaneous values of control surface deflections, engine throttle set- 
ting, relative velocity and direction of the air mass, and inertial angular 
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rates. In some cases, air mass acceleration is included using angle of attack 
rate, a. For our purposes we will assume that these relationships are primar- 
ily linear (except in the case of the rotary terms p, q, r where total veloc- 
ity is used as a divisor to obtain non-dimensional coefficients). 

Thus, neglecting terms which are typically close to zero, the six nondi- 
mensional coefficients are assumed to be in the following form. 


CXB - C Xq «q + C X<x *a + K x + l C X ^ 6i (5-13) 

CYB = Cyg 8 + y v [Cy p P + Cy r r] + Ky Cyg 6i (5-14) 

CZB = C Za a + fy [C Zq q] + K z + £ C z5 .Si (5-15) 

CLB = CLg 6 + [CLp p + CL r r] + J (5-16) 

CMB = C Ma a + [C Mq q] + % + £ C M{ . (5-17) 

i i 

CNB = C Ne e + [C Nr r + C Np p] + % + £ C^i (5-18) 


where, 

V = total velocity (f/s) 
a - angle of attack (radians) 
g = sideslip angle (radians) 
p = roll rate (r/s) 
q = pitch rate (r/s) 
r = yaw rate (r/s) 
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and where 5i represents the 11 control elements including left and right aile- 
rons, stabilizers, elevators, spoilers, throttles, and a rudder. All control 
element values are measured in degrees except throttles which are measured in 
kilopounds of thrust. Appendix A provides a derivative of the coefficients in 
Eqs . 5-13 to 5-18 from linear models. 

Thus, the three translational residuals are 


1 

4 * 

It 

X/m 

(5-19) 

Vy = A y m - 

Y/m 

(5-20) 

v z = A z m - 

Z/m 

(5-21) 


where A x m , A y m , A z m are the c.g. -centered body-referenced accelerometer mea- 
surements, and X, Y, Z are estimates of the aerodynamic forces (also c.g.- 
centered body-referenced) obtained by using measured values of the states and 
control elements in Eqs. 5-7 - 5-9, and 5-13 - 5-15. The aircraft mass is 
denoted by M. 

The rotational dynamic relations (neglecting cross products of inertia) 


are : 


I x P + (I z - x y> RQ = L 
Iy Q + (I x - I Z ) PR = M 
I z R + (I y - l x ) QP - N 


(5-22) 

(5-23) 

(5-24) 


where I x , I y , I z = moments of inertia about the body axes (slug - ft^). 
Three estimates of angular acceleration are then 
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(5-25) 




(5-26) 




(5-27) 


where L, M, N are obtained from using measured values in Eqs. 5-10 - 5-12 and 
5-16 - 5-18. 

Using the trapezoidal rule for approximating the integrals of Eqs. 5-25 
5-27 over a single time interval of A results in the residual equations. 


v p « p(k) - [p(k-l) + (p(k) + p(k-l) ] (5-28) 

and similarly for q and r. Note that if Eq. 5-28 was scaled by 1/ A, these 
residuals will have units of angular acceleration (r/s 2 ). 

Finally, for completeness, we note that preprocessing of the accelerom- 
eter data and a and 3 vane measurements may be necessary to obtain the c.g. 
centered, body-referenced measurements required in the above. Letting 
(£ x , &y» £ z ) denote the coordinates of any sensor in the desired c.g. centered 
body axis coordinate frame, then accelerometer compensation takes the form, 

A x Cora P = A x ra - [“£ x (r 2 + q 2 ) + £ y (pq - r) + £ z (pr + <01 (5-29) 

AyComp s Ay m - [£ x (pq + r) - £ y (p 2 + r 2 ) + Z z (rq - p)] (5-30) 

A z Comp = A Z IU - U x (rp - q) + £ y (rq + p) ~l z (q 2 + p 2 )] (5-31) 
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Compensation of the a and 3 vane measurements takes the form, 

a Corap = a m + j^Cq/V) - * y (p/V) (5-32) 

gComp = pm _ g^Cr/V) - £ z (p/V) (5-33) 

Derivation of these equations appears in [ 2 9 ] • 

Equations 5-19 through 5-21 and 5-28 and the associated roational dynamics 
and force and moment balances provide the information necessary to generate the 
six desired residuals. Unfortunately, for this project, the nondimens ional 
coefficients of the B-737 aircraft were not available. Linear models were 
available, however. Recognizing that there is a one-to-one correspondence 
between the nondimens ional coefficients in Eqs. 5-13 - 5-18 and any linear 
model, it is possible to derive the nondimens ional coefficients from these 
linear models. This derivation is given in Appendix A along with a comparison 
of actual nondimensional coefficient values for two flight conditions. 

5.1.2 Detectability of Aircraft Path Failures Using Flight Test Data 

The availability of flight test data from NASA's B-737 aircraft provided 
a unique opportunity for realistic evaluation of the feasibility of performing 
control element FDI using aerodynamic models of the aircraft. This is because 
many errors which are difficult to simulate are excited during actual flight 
(e.g. wing bending, nonstationary inputs) and realistic values of errors which 
are normally accounted for (e.g. sensor noise) are present. Unfortunately, 
sufficient time was not available to perform a complete analysis. Such 
analysis would require a flight test program to guarantee that all important 
error sources were excited over a full range of operating conditions. Thus, 
this section is meant to demonstrate the application of the concepts developed 
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in Section 4. The results which follow should be considered optimistic due to 
the fact that only a limited amount of data was processed. In particular, we 
used only the data from the first 50 seconds of TSRV flight R380, Run 12JR. 
During this interval the aircraft performed a mild turn maneuver at approxi- 
mately constant altitude and throttle setting. Mild turbulence is believed to 
have been present. Figures 5— 2a to 5— 2q show traces of interesting quanti- 
ties for this maneuver. The flight condition used to obtain linear models 
corresponding to this data is defined by: 

V « 160 KIAS 
h = 3500 ft 
Gear up 
Flaps = 15° 

Y = 0 

The evaluation of detectability presented here is aimed at evaluation of 
limits to performance using the decentralized residuals described in 5.1.1. 
Although the general framework presented in Section 4 would, in principle, 
allow an assessment which is independent of residual generation mechanization, 
such an assessment would be much more difficult due to the complex nature of 
the effects of failures on the measured quantities. In contrast, the effects 
of failures on decentralized residual is easy to characterize resulting in a 
greatly simplified analysis. Furthermore, decentralized residual generation 
was required for further development and this analysis proved useful in the 
FD1 design procedure. 

The basic premise behind the results of this section is that the residu- 
als could be characterized by the alternate failure (and no failure) hypotheses 
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Figure 5-2 (a-q). Flight Data From TYRV R380, Run 12.JR 
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Figure 5-2 (a-q) (Continued). Flight Data From TYRV R380, Run 12JR 
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described in Equation 4-21. That is, failures show up in fixed directions, 
modulated by a failure signature, and the errors could be described by a 
colored noise process whose statistical behavior is independent of the system 
status (i.e., the same for all hypotheses). This latter property is critical 
in enabling us to infer the statistical behavior of the residuals under failed 
modes from the data obtained from normal flight. To see that this is a rea- 
sonable assumption consider a model for the decentralized residual of the 
form. 


p a 

v(k) = no( k ) + I (C i -C i )6 i E (k) + Ci[6i E (k) - 6i m (k)] (5-34) 

i=l 

where r \ o is a random noise term and where, for each of the p control elements, 

A 

S± m is the measured deflection, Si E is the effective deflection, is the 
vector of dimensional control derivatives used in generating residuals, and 
is the actual dimensional control derivative. Under the j tk failure hypothe- 
sis (j=0 indicates no failure) we have 

6i E = 6i m for i * j (5-35) 

6i E t 6j_ m for i = j (5-36) 

The random noise term, tio> takes into account the effects of sensor noise as 
well as modeling error in the formation of the residuals and is, therefore, in 
general, a nonwhite (and generally nonstationary) process which must be char- 
acterized. The term (6i E - 8± m ) in Eq. 5-34 is termed the failure signature 
and is denoted by f^(k). The failure signature is important because when it 
is large, failures are observable. Furthermore, when f^(k) I s large , the 
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undesirable effect on the aircraft is most pronounced so that we must have a 

a 

control system which is tolerant of the resulting disturbance vector f*(k) 
when fi(k) is small. The results of this section quantify "small" in terms of 
failure detectability. 

Let us now define the behavior of the residuals under no failure by a 
stochastic process, t|(k), viz., 

P 

H 0 : v(k) m n(k) = £ (C^-Ci) 5i m (k) + no(k) (5-37) 

i=l 

A simplifying approximation to Eq. 5-34 for the i c ^ failure hypothesis (i*0), 
is then, 

A 

v(k) = n(k) + Ci f i(k) (5-38) 

Equation 5-38 assumes that the statistical behavior of 6i E , under the i th 
failure hypothesis, is the same as Sj. m and that this statistical behavior is 
independent of the failure mode. Clearly, there are many specific failure 
mechanisms in which this is not the case. The assumption is nevertheless, 
very useful in our general failure mode context. For any specific failure 
mechanism, one must decide whether the results derived here are conservative 
or optimistic due to this approximation. The advantage of Eq. 5—38 is that 
the process q(k) is observable from flight recorded data (during normal 
flight) and can, therefore, be realistically characterized. 

Having justified the model of Eq. 4-21 for the decentralized residuals we 
now evaluate the detectability of control element failures using the asymp- 
totic measure B w defined in Section 4. The procedure is as follows. 

1. Compute the six residual signals from Eqs. 5-7 through 5-33 using 
data recorded during flights of the NASA TRSV. 
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2. Compute estimates of the power spectral density (PSD) matrix for the 
six dimensional residual process. 

3. Determine the (dimensionalized) failure directions from the aircraft 
model. 

4. Determine the smallest signature energies, for each control element 
failure, which are required to make specific signature spectra detec 
table. Plot signature energy versus frequency for narrow band signa- 
tures and determine total signature energy for a broadband signature. 

RESIDUAL GENERATION 

Figures 5-3 through 5-8 show the decentralized residuals computed using 
TSRV data sampled at 20 Hz (the figures show a lower sampling rate). Sensor 
compensation (see 5.1.1) was used to accommodate lever arm effects and other 
sensor validation, averaging, and scaling procedures were used (see Appendix C. 
Next to each residual is its sample autocovariance function (ACF) . The sample 
ACF is computed from, 

N-[m | _ _ 

R(m) = 1/N I [ v(n) - v] [v(n+m) - v] t (5-39) 

n=l 

v = 1/N £ v(n) (5-40) 

n=l 

for lags m=0, 1, ... 100. For negative lags we use the property R(m) = R c (-m). 
Note that R(m) is a biased estimate of the underlying ACF but typically has 
smaller mean square error (see [59]). Also, for the 50-second data set sam- 
pled at 20 Hz, N=1000. The ACFs shown in Figs. 5-3 to 5-8 are the diagonal 
elements of R(m). There are several salient features of the residuals which 
are evident from the figures. First, all of the residuals have significantly 
nonzero means. This is expected due to the inaccuracy in predicting the basic 
aerodynamic forces and moments of the airframe and due to the nonzero average 
values of many of the measurements coupled with inaccuracy associated with the 
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Figure 5-3. Translational X-Residual and ACF 
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Figure 5-4, Translational Y-Residual and ACF 
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Figure 3-5. Translational Z-Residual and ACF 
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Figure 5-6. 

Rotational P-Residual and ACF 
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Rotational Q- Residual and ACF 
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Figure 5-8. Rotational R-Residual and ACF 
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corresponding coefficients. Secondly, there is a nearly white component as 
evidenced by the near discontinuity at m=0. This component is due to elec- 
tronic sensor noise and due to discretization error. The third error which is 
evident is a high frequency oscillatory error. The frequency of this error 
differs between residuals and may be due to vibration of the inertial plat- 
form. The oscillation frequencies vary from about 4 Hz to less than 10 Hz. 

Wing bending probably plays no part in these errors since the lower frequency 
oscillations are in the x and y residuals. In the roll residual, a damped 
oscillatory error at about 3 Hz may be due to wing bending. Finally, in many 
of the residuals, one or two first-order-like errors are evident. The time 
constants of these errors are around one second. These errors are almost cer- 
tainly due to the excitation of parametric errors in the aerodynamic model by 
the (forced) motion of the aircraft. 

POWER SPECTRAL ESTIMATION 

The qualitative modal analysis discussed in the previous paragraph sug- 
gests that the estimation of the spectral density matrix ought to be accom- 
plished using some of the more advanced time series methods (e.g., see [54]). 
However, time did not permit full investigation of these methods for the vec- 
tor case. Therefore, spectral densities were estimated using the Blackmun- 
Tukey procedure (i.e., the discrete Fourier transform, DFT, of R(m)). That is, 

100 

S n (u> = l R(m) e"J“« (5-41) 

M=-100 

for me [0, 2x]. Note that S n is, in general, complex (except for the diagonal 
elements which must be real). Also we have the property S n (m+ir) = S n (uriO H 
where H denotes complex conjugate transpose. Figures 5-9 to 5-14 show the 
diagonal elements of S n . 
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Figure 5-13. Spectrum of q Residual 
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Figure 5-14. Spectrum of r Residual 
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The residuals we have generated typically have a large nonzero mean over 
any meaningful interval of interest. It is typical, however, to assume that 
this mean is zero when estimating (or modeling) power spectra. The tacit 
assumption is that for any interval of data, one can eliminate the mean value 
of the process and characterize only the deviations from that mean (assuming 
that the measured mean is actually the statistical mean). However, for our 
purposes we wish to characterize the low frequency energy which is indicated by 
the presence of the nonzero mean in the residuals. In the estimation of the 
autocorrelation function, it is typical to either remove the mean and then add 
in v vT to R(m) or to perform what is known as a circular correlation [55] 
(i.e., assume that the observed sequence is periodic such that v(k+N) = v(k)). 
Both of these approaches address the problem that, for large nonzero mean 
processes, the computations given for R(m) in Eq. 5-40 result in a triangular 
autocorrelation sequence (which does not accurately reflect the, presumed, 
theoretical sequence) and, that because we only have estimates of the autocor- 
relation over a finite window, the resulting power spectral estimates may have 
negative values near (and sometimes not so near) zero frequency when v is 
large. 

To get around these problems and still characterize the very low fre- 
quency behavior of the residuals, we assume that the observed mean value 
really represents a single sample from a filtered low frequency noise 
sequence. That is, since 

_ 1 N 

v = - l v(j) (5-42) 

N j = i 

we know that most of the high frequency energy in v will not have an impact on 
v when N is large. If we assume that the only component of v which can effect 
v has a spectrum of 
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Slf( 0) = G(l-a 2 )/ {(1-a cos( 6)) 2 + (a sin(0)) 2 } (5-43) 

then one can show that E{v 2 } is approximately equal to G as long as -ln(a) « 
2ir/N. Now since we have only one sample of \> (for each window over which we 
wish to estimate the power spectrum) it is difficult to compute a meaningful 
statistical average. Therefore, we will assume that the observed value of v 
represents a typical value and model the low frequency energy in each residual 
using Eq. 5-43 with G = ~v 2 . In practice, we have chosen -ln(a) = 0.6/N. Note 
Note that with typical values of v and N, S L p(0) is very large; however, this 
is consistent with the expectation that for very low frequency noise, signals 
with very large total energy will be needed to achieve detectability. The 
spectra S 4 O) are then added to the spectrum in Figs. 5-8 to 5-13 to evaluate 

detectability. 

FAILURE DIRECTIONS 

A 

The failure directions, C 1# of Eq. 5-38 are given by the dimensional 
derivatives of each control element. Defining the six dimensional residual 


vector as v^- - (v x , 

v y» v z > 

Vp > v q > v r ) > 

failure directions are given by. 


X 

y 

z 

P 

q 

r 

Left Stabilator 

.21 E-l 

.0 

-.25 

.12 E-l 

-.36 E-l 

.88 E-3 

Right Stabilator 

.21 E-l 

.0 

-.25 

-.12 E-l 

-.36 E-l 

-.88 E-3 

Rudder 

.0 

.20 

.0 

.15 E-l 

.0 

-.17 E-l 

Left Elevator 

.99 E-2 

.0 

-.12 

.56 E-2 

-.17 E-l 

.51 E-3 

Right Elevator 

.99 E-2 

.0 

-.12 

-.56 E-2 

-.17 E-l 

-.51 E-3 

Left Aileron 

.11 E-l 

.18 E-2 

-.13 

.12 E-l 

-.47 E-2 

.96 E-3 

Right Aileron 

.11 E-l 

-.18 E-2 

-. 13 

-.12 E-l 

-.47 E-2 

-.96 E-3 

where the numbers 

are given 

in acceleration 

units per degree of deflection. 
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DETECTABILITY EVALUATION 

To evaluate detectability, we considered two types of failure signatures: 
broad-band and narrow band. For narrow-band failure signatures, we can use Eq. 
4-33 to determine the minimum signature energy needed to achieve some desired 
value of B w for various frequencies of excitation. Figures 5-15 through 5-21 
plot E m ^ n versus frequency for each control element using B^ ** 3 and the power 
spectral density estimates described above. As expected, large amounts of 
energy at low frequencies are needed because of the large mean values for the 
residuals. The plots also indicate that aileron and elevator failures may need 
large signature energies at high frequencies. This is probably due to the high 
frequency errors evidenced in the roll residual (v p ). 



FREQUENCY R-33*» 


Figure 5—15. Detectability of Narrow Band, Left Stabilator 
Failure Signatures Versus Frequency 
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Figure 5-16. Detectability of Narrow Band, Right Stabilator 
Failure Signatures Versus Frequency 



Figure 5-17. Detectability of Narrow Band, Rudder Failure 
Signatures Versus Frequency 
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Figure 5-18. Detectability of Narrow Band, Left Elevator 
Failure Signatures Versus Frequency 



FREQUENCY A' 3394 * 


Figure 5-19. Detectability of Narrow Band, Right Elevator 
Failure Signatures Versus Frequency 
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figure 5-20. 


Detectability of Narrow Band, Left Aileron 
Failure Signatures Versus Frequency 



Figure 5-21. Detectability of Narrow Band, Right Aileron 
Failure Signatures Versus Frequency 
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To examine broadband failures signatures, we assume that the failure 
signature could be represented by, 

Bj(u) = (1 ~ &2) (5-44) 

(1-a cos oj) 2 + (a sin to) 2 


By using Eq. 5-44 in Eq. 4-33 and performing the required integrations numeri- 
cally, we can determine the minimum broadband signature energies required for 
detection. Table 5-2 shows the results with a = .4 (~ 8 r/s bandwidth) and 
B w <j = f )m Notice that the results are not symmetrical with respect to surfaces 
on either side of the airplane. This is most likely due to the fact that the 
errors which give rise to our power spectral estimates were not symmetrically 
excited during the interval of data we used. 


TABLE 5-2. DETECTABILITY OF 


Control Element 

Left Stabilator 
Right Stabilator 
Rudder 

Left Elevator 
Right Elevator 
Left Aileron 
Right Aileron 


BROADBAND FAILURE SIGNATURES 

Total Signature 
Energy (degrees) 

0.44 

0.45 

0.62 

0.95 

0.98 

1.85 

1.92 


The values in Table 5-2 are notably small since they represent total sig- 
nature energies (i.e., the integral of the squared signature over time ). Even 
signatures whose total energies are factor of 10 greater than those in Table 
5-2 would be acceptable. A signature energy of 10 degrees can be achieved by 
a constant deflection of 0.5 degrees for one second at the 20 Hz sample rate. 
However, these numbers should only be treated as lower limits to performance 
(best case) for a very specific failure signature which is known and for a 
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perfectly known power spectrum. A more in-depth analysis might include pulse- 
like signatures since they may be more representative of actual signatures and 
might utilize PSD estimate from a broader set of flight data. Detectability 
in terms of pulse height versus pulse width could also be plotted. Further- 
more, distinguishability of failures should be determined using the methods 
described in subsection 4.2.2. 

5.1.3 Aircraft Path Decision Process Design 

The goals and assumptions to be used in this design were detailed in 
Section 2. These included: 

1. No multiple failures 

2. Single flight condition 

3. No additional aerodynamic failure effects 

4. No detailed assumptions about failure signature models 

5. Validated sensors 

The control elements we will consider as possible failures in this design are 
left and right engines (LT and RT), ailerons (LA and RA) , horizontal tail (LHT 
and RHT) , and rudder (R). The horizontal tail is a fictitious surface which 
represents both stabilizer and elevator surfaces. This is done because our 
preliminary evaluation of distinguishability indicated that same— side elevator 
and stabilizer surfaces are indistinguishable based on force and moment bal- 
ance magnitudes along (i.e., without using temporal signature information). 
Aircraft path engine failures are those occurring outboard of the "engine 
actuator" output measurement. Since we have chosen (Section 3.2) the engine 
actuator path as the path from tt\e throttle command to the engine pressure 
ratio (EPR) , aircraft path engine failures are those in which EPR follows the 
throttle command but the resultant thrust is inconsistent with EPR (e.g., 
thrust reversers deployed). 
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ALGORITHM STRUCTURE 

The overall structure of the decision process will include trigger, ver- 
ify, and isolate test procedures as discussed in Section 3.1. This structure 
is chosen as a trade-off between computational complexity and performance in 
solving the unknown onset time problem. For the aircraft path control element 
FDI problem, this structure takes the form shown in Fig. 5-22. The decentral- 
ized residuals defined in subsection 5.1.1 are the only inputs to this deci- 
sion process. All decisions are, therefore, based on the relative sizes and 
spectral characteristics of these six residuals. The trigger process is a 
system monitor which is used to indicate the possible presence of abnormal 
behavior. In order to maximize the sensitivity of the trigger process to 
individual control element failures, separate trigger tests are used for each 
element. That is, each test is designed so that iT_ a detectable failure of 
control i occurs, then its corresponding trigger test will "pass." Note that 
the conclusion that control i is failed if its corresponding trigger passes is 
invalid since such a conclusion is based on the converse of the previous sen- 
tence. One consequence of this structure is that it is possible to have 
redundancy in the trigger tests. For example, if a single residual is the 
only source of information for several triggers, then the resulting trigger 
tests will be all identical. 

The verify and isolate processes are initiated when a trigger test passes. 
The verify process is initiated by the trigger process and is used to reject 
false triggers. The verify process, therefore, plays a major role in achiev- 
ing the desired overall false alarm rate. The isolate process runs in "paral- 
lel" to the verify process and is responsible for making decisions regarding 
the identity of the failed control element. In principle, only the implied 
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Figure 5-22. A/C Path Decision Process 


R-4434 


ambiguity group resulting from a set of triggers (not the union of all trigger 
tests which pass) needs to be "isolated." However, for simplicity, all verify 
and isolate tests will be initiated following any passed trigger test for the 
current design. 

The matrix of verify and isolate tests in Fig. 5-22 indicates that we 
will be using sequential tests which provide "votes" which indicate which of 
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any pair of system hypotheses is more likely (i=0 indicates normal operation; 
the control elements are indexed by i=l, . 7). The full matrix of isola- 
tion test flags, I^j , take on values of 1 when j is more likely than i, 0 when 
i is more likely than j , and 2 when the sequential test has not yet completed. 
When the i,j th test is completed and no decision can be made , I^j and Iji are 
both set to 0. 

To declare that the j 1 -* 1 control element has failed, Vj“l and Iij=l for 
all j*i. That is, the j ttl column of the matrix in Fig. 5-22 must contain all 
l's. Only one such column is possible due to the pairwise comparisons. Other 
information is available from this test matrix and some alternate uses are 
described in [56]. 

HYPOTHESIS TEST DESIGN DETAILS 

The generic decision-process design procedure discussed in subsection 3.1 
will be applied to this problem using the theoretical developments of subsec- 
tion 3.3 The first step in this procedure is to develop structures for the 
hypothesis tests indicated in Fig. 5-22 based on a design model. For this 
model, we will assume that over the time interval needed to detect and iden- 
tify failures, the failure signature is coherent (i.e. , of one sign) and that 
this interval is substantially shorter than the time constants of some assumed 
very low frequency residual errors. We also assume (for defining test struc- 
tures) that another residual error is broadband (i.e. , white) noise associated 
with sensor noise and discretization errors. These errors will be modeled as 
Gaussian processes whose covariance characteristics are the same under all 
modes of operation. 
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After defining the hypothesis test structures from the above design model, 
the parameters of those tests must be chosen. To do this, we will hypothesize 
a statistical truth model for the residual error which is more detailed than 
the design model description. This model is then used to optimize parameters 
and perform sensitivity analyses. In general, such a model might take the 
form, 

x r (k) = A r x r (k-l) + B r w r (k-l) (5-45) 

v(k) = C r x r (k) + D r w r (k) (5-46) 

where v(k) is the six dimensional residual, x r is the n r dimensional "truth- 
state" vector and w r is a white noise vector with covariance matrix Q r . In 
the design procedures to be discussed below, we will assume that the residuals 
are independent and that each residual can be represented by the sum of a 
white noise term, n,,, a first order low pass markov process, n^, and a first 
order high pass Markov process, n^. The white term represents sensor noise 
and discretizaton errors. The low pass term represents "in-band" errors (i.e. , 
those errors which are excited in the same frequency range as the failure sig- 
natures). The high pass term represents unmodeled dynamics. Thus we have. 


Vj(k) = ^^(k) + n^j(k) + nflj(k) (5-47) 

x L j(k) = aL-jx L j(k-l) + (l-a L j )w L (k-l) (5-48) 

x H 3(k) = a^jx^jCk-l) + ( l-a H J )w H (k-l ) (5-49) 

nL-3(k) = XL^(k) (5-50) 

n H J(k) = vqpj (k) - x H j(k) (5-51) 
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The parameters of this model (cutoff frequencies and noise variances) are 
computed from an error budget which is described in subsection 5.1.4. 

Prefilters - The use of colored, stochastic residual errors in the design 
model indicates that a "prewhit ener" is necessary for each hypothesis test 
[32]. Ideally, this prewhitener is a multivariable filter which whitens and 
decorrelates the residual process (note that we don f t really expect to be able 
to achieve a white process in reality since the residual errors are only mod- 
eled as stochastic; the true errors are, of course, nonstationary). While a 
multivariable filter may represent an optimal procedure for the design model, 
it, in general, has one major drawback for the current problem. In general, a 
multivariable filter can (depending on the nature of the frequency errors) 
take residuals which have failures that show up in fixed directions in resid- 
ual space and transform them into signals which have failures that show up in 
time varying directions depending on the temporal characteristics of failure 
signatures. To retain the desired insensitivity to specific failure signa- 
tures, we will first project the residuals into a subspace which is appropri- 
ate for each test (to be based, in part, on the fixed failure directions) and 
then perform a one dimensional whitening operation. This structure will sac- 
rifice performance in comparison to an optimal algorithm when the frequency 
characteristics of the residuals are substantially disparate, but guarantees 
insensitivity of every test to the detailed failure signature characteristics 
(note that results in the previous section indicate that residual errors are 
very similar and so little performance degradation is expected). 

To choose the characteristics of the one dimensional "whitener" for each 
test, first note that the design model which includes low frequency and broad- 
band errors implies that a filter with a high pass characteristic is needed. 
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Rather than hypothesize some detailed power spectrum and solve Weiner - Hopf 
equations [32] (or equivalently design Kalman filters for some detailed state 
space residual model), we will simply choose a high pass filter (HPF) struc- 
ture and select its cutoff frequency. To do this, first note that the cutoff 
frequency must be higher than the bandwidth of the low frequency errors of the 
residuals (the very low frequency errors which vary with, e.g. , velocity 
changes). Secondly, the time constant of the HPF must be larger than the FDI 
interval (so that the assumed coherent failure signatures are not washed-out). 
As a first cut, we will assume that all six residual errors are the same. The 
(very) low frequency error bandwidth is probably no higher than 0.1 r/s. The 
failure signatures are probably not coherent for more than about 2-5 seconds 
and the desired (longest) FDI interval is about two seconds which is consis- 
tent with the length of time that the signature is coherent. As a result, a 
first order high pass filter with cutoff frequency of 0.5 r/s is preliminarily 
chosen for each test. This frequency may change for tests which require 
longer FDI intervals (e.g., because the signatures may have only lower fre- 
quency content). 

Triggers - The structure for the trigger test is based on the projection- 
HPF structure discussed above, and 

1. the assumption that the projected/filtered residual error is white, 
and 

2. the assumption that the projected/filtered failure signature is 
constant. 

These assumptions are reasonable since we assumed that the failure signature 
is coherent and that the time constant of the high pass filter is much longer 
than the FDI interval. The resulting test takes the form. 
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k— N x +1 

S x - j y(j) t x (5-52) 

j=k 


which is just a moving window average of the projected and high-pass filtered 
residuals. This is shown in Fig. 5-23. The absolute value function takes 
into account the uncertain failure sign (see subsection 3.3). As discussed in 
subsection 3.3.1, this test will reliably detect coherent failures of some 
minimal magnitude and all failures which are larger than this minimal value. 
The parameters of this test must now be designed. For these purposes, it is 
convenient to think of the HPF as coming before the projection operation in 
Fig. 5-23 since the first cut design has all HPFs (i.e., for each test) 
identical. 


vOO 



TRIGGER FLAG 
FOR i'th 
FAILURE 


R-3387A 


Figure 5-23. Structure of Trigger Tests - Time Variant 

The parameters of the trigger test are the projection operator, the win- 
dow length, N x , and the trigger threshold. The selection of these parameters 
are based on the following concerns. The projection vector, Pj, must be maxi- 
mally sensitive to the i tl1 failure direction while reducing the effect of real 
errors (i.e., those in a truth model) on S x . Thus, the choice of P£ depends 
on the choice of N x and a statistical model of the filtered residuals. The 
threshold, t x , must be chosen to achieve a desired false trigger rate and it 
is, therefore, a function of P^, N x , and the statistical model. The window 
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length should be as short as possible to minimize trigger delay, but should be 
long enough so that important failures signatures will be detected. The spec- 
ification of minimally detectable signatures is dependent on the choice of Pp, 
N-p, and t-p and the assumed statistical model. 

Given Eqs. 5-47 through 5-51, the statistics of S T can easily be computed 
as a function of Pp and N T and the tradeoff between the false trigger likeli- 
hood and minimally detectable (bias) failure for various values of t T deter- 
mined. To select these parameters we proceed as follows. 

1. Choose N-p as the longest allowable trigger time for the i th failure. 

2. Choose Pp to maximize the distinguishability of the i th failure by 
maximizing the d 2 metric for the random variable S T as a function of 
Pp assuming that the failure direction is Cp and that the HPFed sig 
nature is a constant. That is, compute the mean and variance of Sp 
under the hypotheses H 0 and Hp from 


E(S T | H fl ) = 0 

E(S t I Hp) = NpPptCpfs 

Var (S T | H 0 , Hp) = PptlfPi 


(5-53) 

(5-54) 

(5-55) 


k-Nj+1 

where If is the variance of ( £ v(j)), and f s is the magnitude of 

j =k . 

the constant failure signature (in units of the corresponding control 
element).* Using the results of subsection 3.3, we have 

Pi = Ef^Ci (5-56) 


*Ef can be computed by forming a state variable equation for the (vector) fil 
ter corresponding to the desired sum and solving the discrete time Lyapunov 
equation [57], This can be done by component by components when the compo- 
nents of v(j) are uncorrelated. 
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3. Select the threshold to achieve a desired false trigger specifica- 
tion. For example, to achieve Pp X smaller than 10 -3 , we need 

t x = 3 Var(S T ). (5-57) 

4. Find the value of f s which achieves some specified miss probability. 
For example, to achieve smaller than 10 -3 , we need 

(E(S X | H X > - t x ) = 3 Var(S x ) (5-58) 

5. Determine which elements of the projection vector can be made equal 
to zero. This step is designed to enhance the robustness of the 
algorithm in the case where actual errors are larger than the truth 
model. The idea is to remove those residuals which contribute only 
marginally to detectability as measured by the size of f s needed to 
achieve some Pmt* The procedure is iterative. The projection is 
restricted to have nonzero elements in only a subset of residuals and 
steps one to four are carried out. If f s does not increase substan- 
tially (e.g. , no more than one percent greater than its value with 
all residuals) then the residuals which are not included do not con- 
tribute to detectability and can be removed from that test. 

The five steps just described provide a precise methodology for determin- 
ing the values of the parameters of the trigger test in Fig. 5-23. Of course, 
these values depend highly on the statistical truth model assumptions, and, 
therefore, one must be careful in interpreting the results for nonstationary 
errors. For example, if Eqs. 5-48 to 5-52 are derived by considering observa- 
tions of real errors during many different flying modes, a high degree of con- 
fidence in the resulting design may be allowed. In the remaining designing 
methodologies, we will also utilize Eqs. 5-48 to 5-52, and, therefore, the 
same comments apply. 


Verify Test Design - The structure of the verify test is similar to the 
trigger test in that a "prewhitening" high-pass filter, projection operation 
occurs first. Assuming once again that the projected and filtered residual is 
zero mean and white during normal operation and nonzero mean with the same 
additive white noise during a failure, the optimal sequential probability 
ratio test takes the form, 
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tc k. 

S v (k) = | I G AY (j)| - l G D > t v > Verify passes 

j=k f j=k f 

< -ty > Verify fails (5-59) 

The test is started when a trigger occurs (i.e., at time k£) and completes 
under the conditions in Eq. 5-59 or when a time limit, N v , is reached. Figure 
5-24 illustrates this test. 

VERIFY FLAG 
FOR i'th 
FAILURE MODE 

R- 3388A 

Figure 5-24. Structure of Sequential Verify Tests 

The parameters of the verify test which must be chosen are Pi, G A , Gj), 
t v , and N v » For a given value of t v , G D controls the speed at which false 
triggers are rejected and G A controls the speed at which failures are detec- 
ted. The relationship between G A and Gp determines what failure sizes will 
tend to result in a passed verify test. As discussed in subsection 3.3.1, 
failure signatures greater than some minimally detectable value will be detec- 
ted in shorter times by this test. As in the trigger test. Pi is chosen to 
maximize the sensitivity to failure i and minimize the effect of residual 
errors on the test. The tradeoffs which must be made in the design of a 
sequential test are similar to those in fixed sample size tests (Pfa> atld P MD 
versus failure size) except that we must include a probability of making no 
decision for tests which end in such a conclusion at their time limits. 
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Unfortunately, as we have indicated, the relationship between these probabili- 
ties and the test design parameters is very difficult to compute when there is 
a mismatch between the design and truth models. Thus, we must make use of 
heuristic methods for test design. One such method was discussed in subsec- 
tion 3.3 and will be expanded here. The method proceeds as follows. Note 
that all expectatons are taken with respect to the pdfs defined by the truth 


model given in Eqs. 5-47 to 5—51. 

1. Choose a maximum sample length Ny. Since the verify process is 
designed to validate triggers, N v - N T is a reasonable choice. 

2. Choose Pi to maximize the distinguishability of the i th failure by 
maximizing the d 2 metric for the random variable Sy(Ny) as a function 
of Pi assuming that the failure direction is Ci and that the HPFed 
signature is a constant. Note that unlike the trigger test we must 
specify a particular time at which d 2 is evaluated because the sequen- 
tial test is a time varying test (i.e., it is triggered). The equa- 
tions for choosing Pi are identical to those for the trigger test and 
if Ny - Nj, then the value of Pi is also the same. 

3. Find the smallest value of E(y|H^) which makes the fixed sample size 
test of length Ny reliable; where reliability is measured in terms of 

(large values of d 2 imply high reliability, see subseciton 3.3.1). 
Set equal to this value and set Gq = G^ 2 / 2. This choice implies 
that failure signatures which result in average values of y which are 
larger than G A /2 tend to drive the sequential test to its positive 
threshold and those less than Gy^/2 tend to drive it to its negative 
threshold. 

4. Determine thresholds which result in low probabilities of false alarm, 

missing a minimally detectable failure, and making no decision. To 
accomplish this, consider the ideal SPRT. Let the desired detectable 
mean be m, and the process, y, be white with variance a 2 . Then from 
[33] we have G^ = m/a 2 , Gp * m/(2a 2 ), and ty = -Ln[ (P m)/(1“Pf) 1 > where 
P M and Pp are upper bounds on the desired missed detection and false 
alarm probabilities. An interesting point in the test is when 

E(Sy(k)|Hi) equals ty (note that the symmetry of the test implies that 

this is the same point as when E(Sy(k)|H 0 ) equals — ty). Denote the 
value of k for which this is true by k t . Then it is easy to show that 

the d 2 metric between the hypothesis of zero mean and the hypothesis 

of mean m (with additive white noise) for Sy(k^) is just 2ty. We now 
use this information to select thresholds for test Eq. 5—59 using the 
truth model distribution. 
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4a) Find k t such that d 2 for Sy(k t ) = -2Ln[(PM)/(H?F) ] (assume that 
the parameters of the test are given by steps two and three and 
that the hypothesis % includes a mean of minimally detectable 
magnitude; i.e., E(y|Hi> = 

4b) Set ty to make E(Sy(k t )|Hi) = -E(Sy(k t ) |H 0 ) = ty, viz. 
ty = k t Ga. 2 / 2 = k t Gj). 


5. As in the trigger test, an iterative procedure to zero projection 

elements which contribute little information to the test may follow. 
Note that this may be unnecessary if Ny = N-j- since the projections 
are then the same as those of the trigger test. 


Isolation Test Design - The isolation tests are also started when a trig- 
ggr occurs and completed when either a decision or a time limit, Nj , is 
reached. The isolation tests must have the property that failures which are 
ldrgsr than some minimal value should result in faster decisions. To achieve 
this we will use the failure magnitude invariant tests described in example 
three of subsection 3.3.2. In these tests, the projection operation we need 
to specify is orthogonal to one failure and maximally sensitive to another. 
Ideally, two "rejection" tests for each hypothesis pair would be implemented. 
However, to avoid some computational complexity, we will only perform one test 
for each hypothesis pair. As for the verify tests, the isolate test structure 
is based on the SPRT for detecting a constant in the projected and filtered 
residuals with additive white noise. The structure is shown in Fig. 5-25 and 
takes the form, 


k k 

Si(k) = | l Yi(t)| - | l Yj(t)| > tj > test i over j passes 

t=k f t=kf test j over i fails 


(5-60a) 


< -tj — > test j over i passes 

test i over j fails (5-60b) 
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v(k) 



Figure 5-25. Structure of Sequential Isolation Tests 

The parameters of the isolation test are the two projection operations 
and Pj/i (both assumed to be unit vectors), tj, and the time limit Nj. 

The projections are unit vectors because choosing otherwise would bias the 
test towards detecting smaller mean values in either or yj. (Note, such a 
case may be desirable since the average values of the corresponding control 
element which achieve equal size failures in and yj could be different). 

The projection Pi/j is orthogonal to failure direction j and sensitive to 
failure direction i, and conversely for Pj/i« The threshold, tj , and the time 
limit, Nx, control the error likelihoods and the probability of making no 
decision (since "no decision" will be the conclusion if neither Eq. 5-60a nor 
5-60b is true when k=kf+Nx). The sequential nature of this test, again, makes 
the relationship between the test parameters and the test goals difficult to 
compute. Therefore, a heuristic procedure similar to that used in the verify 
test is used. The procedure is described as follows. 

1* Choose a maximum sample length Nx« 

2. Choose the projection Pi/j to maximize the d 2 metric for the quantity 
kf+Ni 

( £ Yi(t)) subject to the constraint Pi/j c Cj = 0, and similarly 

t=k f 

for Pj/i. 
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3. Find the smallest values of 7 - E( Yi > - E ( Yj ) which make the fixed 
sample size test of length Nj based on Sx(Nx) reliable where reli 
bility is measured in terms of d 2 . To compute d 2 for Sx(Ni) we have, 


E(Sx|H 0 ) = N x s 


(5-61) 


E(Sx|Hj) = -Nxs 


Var(Sx|Hi, Hj) < (Pi/j Tf p i/j> + (p j/i Z f P j/i^ 


(5-62) 


(5-63) 


kf+Ni N N 

where Ef is the covariance of £ v(t), where v(t) is the HPFed 

t=k f 


residual vector, and where the inequality is due n « gl * ct * ng 
covariance between the absolute value terms in Eq. 5 60 (note that 
this inequality may give rise to some conservatism in the design o 
the isolation tests. A tighter bound would substract 2Pi/jEfPj/i 

from 5-13). 

Find the value k t at which d 2 , for Sx(k t ), is equal to -2 Ln[P e / 
(1-P e )], (P e is a lower bound on the desired probability of error). 

Set tx = k t 7 (is E(Si(k t )|Hi or Hj ) = ti for failures which make 

E(yx | H i ) or E(yj|h) equal to s. 

Iterate on steps two through four to determine which residuals con- 
tribute little information to decision reliability. This is done by 
examining the increase in the average values of the failure signature 

required to achieve 7 for both failure modes. That is, let 


f . = s / (Pi/j Ci) 


f j = s / (Pj/i Cj) 


(5-64a) 


(5-64b) 


A residual may be removed from the test (its projection zeroed) if, for exam 
pie, the resulting vslue of «i + f j ) in "» bigger than 1.1 tines its value 

with all residuals considered. 
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5.1.4 Detailed Design and Test Results 

In this subsection, we present a detailed design of an aircraft path con- 
trol element FDI system using the design procedures discussed in subsecion 
5.1.3, evaluate its expected performance and demonstrate its capabilities 
using simulation data from NASA’s nonlinear B-737 simulation. This subsection 
starts with a discussion of error budgets for use in the statistical truth 
model of Eqs. 5-47 to 5-51. Preliminary detectability results based on this 
error budget are computed by designing trigger filters for each of the seven 
control element failures considered as potential failures (left and right 
engines, ailerons and horizontal tails, and the rudder). Horizontal tail is a 
fictitious element used to represent elevator or stabilizer since these sur 
faces are indistinguishable. Several design iterations are then discussed 
qualitatively ending with the final error budget and design figures. Results 
of testing the resulting FDI algorithm using data from NASA's simulation are 
then presented. These results include false alarm checks and detection/isola 
tion checks for various maneuvers and failures and a short investigation of 
alternative algorithm structures and threshold schedules. The subsection then 
concludes with a discussion of the results and conclusions about the FDI algo- 
rithm and the design procedure. 

ERROR BUDGET ANALYSIS 

For the design procedure described in subsection 5.1.3, a statistical 
truth model taking the form of three independent error sources for each resid- 
ual was required. The three error sources consisted of a white noise term, a 
fi rs t order low-frequency term and a first order high-frequency term. An 
error analysis is now used to determine the bandwidths and energies in these 
processes. 


147 




ALPHATECH, INC. 

All of the error analyses to be used here are based on a linearization of 
the residual equations. Table 5-3 shows the dimensional derivatives which 
comprise this model. Constant terms and velocity terms are neglected because 
they only contribute to very low frequency errors which are washed out by the 
"prewhitening" high pass filter. Both elevator and stabilizer surfaces are 
listed separately since they may not always be used proportionally in the con- 
trol law, and spoiler panels are also listed since they may also be used. 


TABLE 5-3. DIMENSIONAL DERIVATIVES 



•(*«*) 

0(deg/s) 

*«*«*/•) 

9(d*g/*) 

r(d«g/*) 

TL/TR(klb*) 

SL/SR(d«g) 

EL/ER(deg) 

AWAR(deg) 

R(d«g) 

SPL/Srk 

W* (ft/. 2 ) 

.57* 

0 

0 

.000309 

0 

.342 

.0211 

.00989 

.0113 

0 

-.0181 

(ft/* 2 ) 


- .663 

.0131 


.0372 




1.00179 

.202 

i.oies 

v x (ft/* 2 ) 

-3.60 



-.00355 


-.00474 

-.248 

-.116 

-.133 

0 

.133 

Vp (r/* 2 ) 


- .0629 

- .0367 


.0124 

1.0021 

1.0119 

1.00556 

1.0116 

.0147 

♦ •013 

v q (r/* 2 ) 

- .0327 



-.0113 


.00620 

-.036 

-.0167 

-.00473 

0 

.00197 

v r (r/* 2 ) 


.0159 

- .00267 


- .00300 

1.0124 

1.000883 

1.000506 

1.000959 

- .0174 

♦ .002 
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White Noise Budget - The white noise term represents errors due to sensor 
noise and discretization of the sensor measurements. The total variance of 
the white noise term is computed by taking the root sum square (rss) of the 
contributions of each measurement to each residual. That is. 



(5-65) 


where Cjj represents the sensitivity of the i th residual to the j ttl measure- 
ment and Oj is the standard deviation of the corresponding measurement noise. 
(Note that accelerometer measurements and angular acceleration computations 
are also taken into account in Eq. 5-65). Table 5-4 shows the sensor noise 
values used in this study. 


TABLE 5-4. 

SENSOR NOISE 

; BUDGET 

Measurement 

a 

Units 

a 

.4 

degrees 

0 

.4 

deg 

P 

CM 

O 

• 

deg/s 

q 

.02 

deg/s 

r 

.02 

deg/s 

TL/TR 

.06 

klbs 

SL/SR 

.1 

deg 

R 

.1 

deg 

EL/ER 

.1 

deg 

AL/AR 

.1 

deg 

SPL/SPR 

.1 

deg 

A x 

.32 

ft/s/s 

Ay 

.32 

ft/s/s 


.32 

ft/s/s 
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The resulting white noise budget (i.e. , the °n w i in Eq. 5-65 with the C^s in 

Table 5-3 and a? of Table 5-4) is shown in Table 5-5. 

J 


TABLE 5-5. 

WHITE NOISE 

BUDGET 

Measurement 

0 

Units 

X 

.39 

ft/s/ s 

Y 

.42 

ft/s/s 

Z 

1.5 

ft/s/s 

P 

.035 

r/s/s 

Q 

.017 

r/s/s 

R 

.012 

r/s/s 


Low Frequency (LF) Budget -The "low" frequency errors to be modeled are 
mostly due to parameter errors; i.e., errors in C^j of Eq. 5-65. These errors 
are modulated by changes in the measurements yj which nwltiply C-y in the 
(linearized) residual generation equations. The first parameter to be speci- 
fied for the LF errors is the cutoff frequency for each residual. The param- 
eters are then determined from 

-“LF^At) 

a^p* = e (5-66) 

where oilf*- are the cutoff frequencies in radians/sec and At is the sample 
time. There are several ways in which the cutoff frequency could be computed. 
For example, examination of the closed-loop transfer functions (TFs) for the 
control law in use would indicate the frequency ranges in which each measure- 
ment may be excited. For the control law in [58], for example, transfer 
functions begin to roll off from anywhere between .2 r/s and 10 r/s. Such an 
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analysis indicates that more than a single LF error term may be needed since 
the different transfer function bandwidths can affect a single residual. In 
spite of this fact, the dominant TF roll off frequency is about 2 r/s. Since 
this is consistent with the covariance functions observed in subsection 5.1.2, 
we will use o^pi = 2 r/s for each residual. 

The magnitude of the LF errors in each residual are specified in terms of 
the total variance of the process n^p*. Since these errors are mainly due to 
parametric errors, we must translate a specification of each parameter error 
into the total variance of the corresponding LF error process. There are sev- 
eral ways in which this can be accomplished, however, in every case, some 
notion of a design envelope is required. 

The design envelope is introduced in order to determine the potential 
size of the contribution of any parametric error to the residuals. The design 
envelope amounts to a specification of the largest variations of each measure- 
ment in the bandwidth of interest. Since the HPF washes out errors below 
.5 r/s and the LF bandwidth is 2 r/s, the bandwidth of interest is [.5,2] r/s. 
Table 5-6 shows the design envelope used in this project. Note that we 
assumed that spoilers are not used in the control law (but are used in the 
residual generation process, thus affecting the white noise budget). These 
numbers were determined from representative simulations of the aircraft 
response to various maneuvers using the control law of [58]. 

The total variance of each LF term is now computed as follows. First 
specify the maximum variation (in percent) for each coefficient j . Find the 
worst case error for each residual and then divide by three to get the total 
standard deviation of the corresponding LF term. That is, 
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TABLE 5-6. DESIGN ENVELOPE 


Measurement 

Max Variation 

Units 

a 

5.0 

degrees 

6 

5.0 

deg 

P 

20. 

deg/s 

q 

10. 

deg/s 

r 

10. 

deg/s 

TL/TR 

3.0 

klbs 

SL/SR 

10. 

deg 

R 

10. 

deg 

EL/ER 

10. 

deg 

AL/AR 

10. 

deg 

SPL/SPR 

0.0 

deg 

Ax 

10. 

ft/s/s 

Ay 

2.0 

ft/s/s 

^z 

15. 

ft/s/s 


On 1 = \ l I AC ij y j max I (5 " 67) 

5 j 

where AC-jj is the worst case error in Cjy and yj max i s the corresponding mea- 
surement's design envelope. The division by three is used because the stand- 
ard deviation represents a typical value and we used a worst case analysis in 
summing errors. Furthermore, the design methodology detailed in 5.1.3 utilized 
measures in which error probabilities are roughly equal to a 3o significance 
level. Thus, it is expected that the worst case LF errors will just barely 
cause decision "errors" (actual errors or no decisions) to be made in the 
hypothesis testing procedures. For five percent errors in every coefficient 
and the design envelope of Table 5-6, the LF residual errors are given in 
Table 5-7. 
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TABLE 5-7 . PRELIMINARY LF ERROR BUDGET (1-gVALUES) 


Measurement 

o 

Units 

X 

0.096 

ft/s/s 

Y 

0.10 

ft/s/s 

Z 

0.47 

ft/s/s 

P 

0.033 

r/s/s 

Q 

0.024 

r/s/s 

R 

0.0077 

r/s/s 


At this point it is worthwhile to compare the values in Table 5-7 to the 
control derivatives in Table 5-3 to get some idea of detectability. For exam- 
ple, we f d expect the pitch residual to play a major role in detecting elevator 
failures* However, in order for the effect of an elevator failure to be reli- 
ably detected, it should be about six times greater than the l-o value of the 

corresponding LF error (d 2 = 6 > P e » lCF 4 ). This would imply a need for a 

an average deflection of 8*6 degrees! Although smaller failures could be 
detected, this is the minimum which achieves the desired reliability. Thus, 
it appears that the design envelope may be too conservative. More will be 
said about this in subsequent discussions. 

High Frequency (HF) Budget - High frequency errors in the residuals may 
be caused by neglected sensor dynamics and neglected flexure modes. For this 
study, we assumed that only the first-asymmetric and first-symmetric wing 
bending modes contribute to high frequency error. Furthermore, we assumed 
that these modes are excited above about 15 r/s. 

The first order high pass Markov process described in subsection 5.1.3 is 
a poor model for the effects of these errors. This is true for two reasons. 
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First, the frequency shape does not correspond to the "peaky" nature of the 
real errors, and second, the power spectrum defined by the cutoff frequency of 
15 r/s (with the sample period of .05 sec.) is virtually the same as any other 
spectrum with cutoff frequency above about 10 r/s. The latter effect is due 
to the specific realization of the HF errors which in the limit (as — > ~) , 
looks like a one time step lag. Due to these shortcomings, and to the lack of 
wing bending models in the simulation to be used for testing purposes, the 
final design will be based on no HF errors. Nevertheless, it is instructive 
to see an example of HF error budget modeling. 

For the first asymmetric wing bending mode, the primary residual affected 
is v p . The dynamic errors associated with this mode are excited mostly by 
differential aileron motion and somewhat by rolling motion due to rudder 
deflection. Assuming that the major contribution is due to differential aile- 
ron, we can define the total variance in the HF error in v p by 

0P HF = “ I c p6A A«A max | 

1 

= - (.012)(20) = 0.08 (5-68) 

3 

Similarly, for the first symmetric wing bending mode, the primary residuals 
which are affected are Vq and v^. The HF errors in both residuals are excited 
by collective aileron deflection and by vertical wind gusts. Assuming that 
vertical wind gusts result in a change of a of no more than five degrees the 
total variance of the resulting HF errors is. 
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= - [ ( .005) (20) + (.033X5)] - .088 (5-69) 

3 

1 max 

o z HF - - I I C Z 6 A I6 a I + I c za a max | ] 

= - [ ( . 13) (20) + (3.6X5)] = 6.9 (5-70) 

3 

DESIGN ITERATIONS 

Several design iterations involving changes to the LF error specifica- 
tions were made before a final specification was utilized in a full scale 
design. A short diary of these iterations is given here. In every design 
iteration it is assumed that the maximum length for a test is one second 
(Noax = 20) and that a first order HPF with cutoff frequency of 0.5 r/s is 
employed as a preprocessor. 

The first design iteration utilized the error budgets developed above to 
design trigger tests. For the one second maximal trigger window length, pro- 
jections were computed, false alarm thresholds were set to achieve (nominally) 
P F = 10 -4 (i.e., at 3a[Sx(20)]), and the average values of control "deflection" 
required for reliable detection (i.e., 1-Pp = 10 -4 for Gaussian statistics) 
were determined. This information is shown in Table 5-8. The projections are 
computed without any attempt to zero useless components (see subsection 5.1.3). 
Also, the aileron projections are incorrect due to a transcription error in 
the design software data. The detectability level is the average value of 
control deflection required for reliable detection (as defined in Table 5-8. 
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TABLE 5-8. AVERAGE VALUES OF CONTROL DEFLECTION REQUIRED TO ACHIEVE 1-Pd= 10 4 
(SIGNATURE "SIZE" 6q [S T (2Q)]) 


CONTROL 


TL/TR 

SL/SR 

EL/ER 

AL/AR 

R 

Projection 

X 

.089 

.013 

.013 

.017 

0 

(For left 
side where 

Y 

0 

0 

0 

.001 

.020 

different) 

Z 

0 

-.006 

-.006 

0 

0 


P 

.012 

.17 

.17 

.16 

.061 


Q 

.068 

-.97 

-.97 

-.96 

0 


R 

.99 

.16 

.16 

.21 

-.99 

DETECTABILITY 

LEVEL 


1.97 klbs 

3.2 deg 

6.4 deg 

6.9 deg 

2.3 deg 

All these values are 

quite large, as 

expected 

from preliminary comparisons of 

control derivatives 

with 

the LF error budget. 

It is deemed that 

these values 


are not satisfactory. 

The only way that detectability levels can be reduced is by restricting 
the envelope of operation such that low frequency errors are reduced. This is 
obviously not a satisfactory solution by itself. However, the use of thresh- 
old scheduling permits us to reduce the nominal envelope and achieve the same 
false trigger alarm rates by scheduling thresholds during periods where errors 
are expected to be large. The resulting detectability levels are, of course, 
only valid for flight within the nominal envelope and deviations from this 
envelope will cause thresholds to be increased and detectability levels to 
increase. 

We first considered eliminating the largest errors from the LF budget. 
This then implies that schedules which are a function of the corresponding 
measurements will be necessary. The largest errors are due to coefficients of 
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1. a for v x 

2. g for Vy 

3. a for v z 

I 

4. g and p for Vp 

5. SL/SR for Vq 

6. r and g for v r 

Setting the errors in these coefficients equal to zero results in a smaller LF 
error budget and detectability levels of 
TL/TR : 1.4 klbs 

SL/SR : 1.7 deg 

EL/ER : 3.4 deg 

AL/AR : 3.6 deg 

R : 1.5 deg 

These detectability figures are more acceptable. 

The last design iteration was accomplished after observation of the size 
of the residuals generated by processing simulation data from NASA's B-737 
simulation. The observed residuals during a mild climbing turn maneuver were 
consistent with the maximum LF (model) errors used in the previous error bud- 
get. However, during roll, pitch and yaw doublets, were substantially (in 
magnitude) larger than what would have been predicted by this budget. 

Since we ignored the largest errors in the previous budget, one explana- 
tion of the observations is that it is these errors which are being excited 
and should be accounted for by threshold scheduling. However, the temporal 
characteristics of the residuals are not correlated with the coefficient 
errors which were ignored in this budget. Thus, some other means of develop- 
ing reasonable error budgets is necessary. 
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To expedite our further work, a final design was created using an error 
budget based on observed errors. Using l-o values corresponding to 2/3 of the 
maximum LF error observed during the climbing turn maneuver, we expect that 
errors which are twice as large as those observed during this maneuver to be 
adequately handled. The standard deviations of the LF errors used in the 
final design are. 


x : 

O 

CM 

• 

ft/s/s 

y : 

.033 

ft/s/s 

z : 

.20 

ft/s/s 

P : 

.020 

r/s/s 

q : 

.0050 

r/s/s 

r : 

.0034 

r/s/s 


These budgets are larger than the previous design iteration for X, Y, Z, and P 
residuals and smaller for Q and R residuals. The resulting reliable detecta- 
bility levels for the triggers (now with useless projection elements zeroed) 
are, 

TL/TR : 1.8 klbs 
SL/SR : 1.0 deg 
EL/ER : 2.0 deg 
AL/AR : 5.6 deg 
R : 1.5 deg 

Table 5-9 shows the "optimized" trigger projections. Notice that for detect- 
ing aileron failures, several residuals are required. This is due to the low 
overall effectiveness of the ailerons and to the fact that their effectiveness 
is spread amongst many axes. The use of only the r residual for rudder detec- 
tion and only the q residual for horizontal tail detection is consistent with 
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TABLE 5-9. OPTIMIZED TRIGGER AND VERIFY PROJECTIONS FOR FINAL BUDGET 


CONTROL 


TL/TR 

HORIZONTAL TAIL 
SL/SR & EL/ER 

AL/AR 

R 

Projection 

X 

.011 

0 

0 

0 

± indicates 

Y 

0 

0 

0 

0 

sign for 
left/right 

Z 

0 

0 

-.006 

0 

controls 

P 

0 

0 

±.189 

0 


Q 

0 

1.0 

-.905 

0 


R 

±1.0 

0 

±.380 

1.0 


qualitative expectations, as is the use of both x and r residuals for aircraft 
path engine failures. 

The procedures for designing verify and isolate tests described in sub- 
section 5.1.3 were followed and projections, thresholds, and distinguishabil- 
ity levels computed. Table 5-10 summarizes the distinguishability results. 
Each entry in the table represents a particular isolation test. The first 
number corresponds to the average control deflection needed for the test to 
"pass" (pass means that the test decides in favor of the control identified by 
the column heading) and the second number corresponds to the average deflec- 
tion needed for the test to "fail" (fail means that the test decides in favor 
of the control identified by the row heading). The largest of these numbers 
represents the overall distinguishability level for each surface and is shown 
at the right in Table 5-10. The tests corresponding to the largest values are 
highlighted. 

Finally we note that the detectability and distinguishability levels are 
substantially larger than those determined in subsection 5.1.2. This is 
because the error budgets used in the design process were substantially larger 
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TABLE 5-10. DISTINGUISHABILITY LEVELS FOR ISOLATION TESTS 


RT 

LHT 

RHT 

R 

LA 

OVERALL 

DISTINGUISH' 

ABILITY 

RA LEVELS 

LT 1.5 / 1.5 

1.8 / .68 

1.8 / .68 

@/@ 

( 0 )/ 3.5 

1.8 / 3.9 

1.9 klbs 

RT 

1.8 / .68 

1.8 / .68 

0/0 

1.8 / 3.9 

( 0 )/ 3.5 

1.9 klbs 

LHT 

— 

@/@ 

.62 / 1.3 

2.4 / (£0) 

1.2 / 5.5 

3.7 deg 

RHT 

— 

— 

.62 / 1.3 

1.2 / 5.5 

2.4 / 6.0 

3.7 deg 

R 

— 

— 

— 

1.4 / 3.4 

1.4 / 3.4 

1.7 deg 

LA 

_ 

« 

— 

— 

4.3 / 4.3 

6.0 deg 


than the observed values and because the FDI algorithm was purposely not 
optimized for the statistical (truth) model used in the evaluation process. 

SIMULATION RESULTS 

Simulations of a modified B-737 aircraft were performed at the NASA 
Langley Research Center and the required data were recorded on magnetic tape 
and delivered to ALPHATECH for use in simulating the FDI algorithm. The NASA 
simulation is a full six degree of freedom nonlinear simulation with nonlinear 
aerodynamic coefficients, accurate actuation models, sensor errors, and turbu- 
lence simulation using the Dryden spectra [59]. A total of 41, 60-second, 
simulation runs were made encompassing many categories of tests. For aircraft- 
path failures, these categories included: 
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1. False alarm checks with three "doublet" maneuvers and a climbing 
turn, with and without turbulence and sensor noise, 

2. Detection checks for totally missing control surfaces, with and 
without turbulence and sensor noise, and 

3. Detection checks for varying degrees of partially missing failures 
with and without turbulence and sensor noise. 

In all of the runs, the control law defined in [58] was used. 

The results to be presented next are for the climbing turn maneuver. 
Discussions of performance during the doublet maneuvers follows. Investiga- 
tions of alternate algorithm structures (threshold scheduling and forced deci- 
sions) are presented next and we conclude with a discussion of the lessons 
learned. 


Climbing Turn Results (No Failure) - Simulation data of the climbing turn 
maneuver with no failure were processed by the FDI algorithm. The climbing 
turn maneuver is accomplished by commanding pitch and bank angles (regulated 

by the control law of [58]). The commands are given by 

• • 

At 10 sec 9 C = 7.5 deg/sec and <J> C = 1.5 deg/sec 

• • 

At 14 sec 9c = ‘frc = 0 

• • 

At 21 sec 9 C = “7.5 deg/sec and <j> c = -1.5 deg/sec 

• • 

At 25 sec 9c = <t>c = 0 

Three cases were examined? no turbulence or sensor noise, sensor noise but no 
turbulence, and both turbulence and sensor noise present. During each of 
these runs, no false trigger was ever recorded. 

The maneuver begins at ten seconds. Figure 5—26 shows the attitude, 
angular rates, and linear accelerations of the aircraft during the maneuver. 
Figures 5-27 through 5-32 show the six residuals for the turbulence and sensor 
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noise case. Figure 5-33 shows the trigger test for the left aileron.* Note 
that an initiation transient due to large low frequency errors causes the 
trigger to cross its threshold, however, an "inhibit" flag prevents decisions 
from being made for the first four seconds (two time constants of the HPFs) of 
operation. Other triggers look similar to Fig. 5-33. Low frequency errors 
which are excited during the maneuver are clearly evident, however, the design 
methodology's selection of thresholds account for the impact of these errors. 


LA TRIG 



TIME (SECS) 


R-4442 


Figure 5-33. LA TRIG 


*The scaling on all hypothesis test plots in this section do not correspond to 
the design data given in the previous subsection. This is because standard 
log-likelihood-ratio software was used to generate the simulation results. 
These log-likelihood-ratio tests are functionally equivalent to the ones 

specified above. 
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Climbing Turn Results (Detection of Totally-Missing Failures )- For these 
results, totally missing failures were similated (Effectiveness = 0 at 5.0 
seconds before the initiation of the climbing turn maneuver at ten seconds). 
The simulations were run without turbulence but with sensor noise. No air- 
craft path engine failures were available. Both elevator and stabilizer fail- 
ures were simulated, although no attempt was made to distinguish between the 
indistinguishable modes (same side elevator and stabilizer). Table 5-11 shows 
the trigger times and isolation times for each failure mode. Note that trig- 
gers for elevator and stabilizer failures occur before the maneuver because 
they are providing nonzero forces and moments to the aircraft during straight 
and level flight. 


TABLE 5-11 TRIGGER AND ISOLATION TIMES FOR TOTALLY MISSING SURFACE FAILURES 


Failure 

Trigger 

Isolate 

Mode 

Time (sec) 

Time (sec 

LS 


6.15 

RS 


6.25 

LA 

10.9 

11.70 

RA 

10.9 

11.55 

R 


11.10 


LE 5.4 (First) Out of time 

(ambiguity group = LHT/RHT) 

RE 5.4 (First) Out of time 

(ambiguity group = LHT/RHT) 
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Figures 5-34 through 5-39 show the residuals for the rudder failure case. 
The trigger, verify, and isolate tests are shown in Figs. 5-40 to 5-47. 

Notice that the last isolation tests to decide are the hardest ones (as judged 

by Table 5-10), namely, R/RT and R/LT. 

The elevator failure modes are not isolated (to the corresponding hori- 
zontal tail mode) during this run. This is due to the fact that the most dif- 
ficult test (LHT/RHT) can not decide by its time limit. Figure 5-48 shows the 
LHT/RHT isolation test during both left and right elevator failures. Notice 
that triggers occur frequently and that in the greater majority of cases, the 
isolation statistics are heading in the correct direction. Examination of the 
elevator position indicates that its average value within the FDI bandwidth 
(.5 r/s to 6 r/s) is about six to seven degrees. Comparing this to the 7.4 
degrees needed for distinguishability (Table 5-10; a factor of two times hori- 
zontal tail requirement due to 50 per cent effectiveness of elevator with 
respect to stabilizer), we see that the totally missing elevator failure for 
this maneuver is only marginally detectable (i.e., the likelihood of missing 
it is larger than desired). To see if, in fact, this failure mode is a criti- 
cal failure. Fig. 5-49 shows the aircraft response during this failure. 
Comparison of Figs. 5-49 and 5-26 indicates that, in spite of the missing 
elevator, the climbing turn maneuver is successfully accomplished with little 
performance degradation. Thus, the qualitative assessment of this failure is 
that, for the control law being used, it is not severe. 

Finally, Table 5-12 shows the trigger and isolate times for varying 
effectiveness levels of left aileron failures. The results are as expected. 
Smaller failures result in longer isolation times. When the failure is small 
enough, triggers occur, but isolation decisions can not be made (in this case 
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(a). LHT/RHT Isolation Test, Left Elevator Failure 


J — 

LHT 

1 

1 

1 

1 1 

THRESHOLD 

A, . . 



^ A- 

\ 

RHT 

* vri ^ 


n 

v V 

THRESHOLD 
! 1 

L_ 

1 

1 ! ! 


5.0 10.0 15.0 20.0 25.0 30.0 35.0 40.0 

TIME (SECS) 


R-4458 


(b). LHT/RHT Isolation Test, Right Elevator Failure 
Figure 5-48. 3/4 
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due to the inability to verify the failure). When the failure is exception- 
ally small, no triggers are recorded. 

TABLE 5-12. TRIGGER AND ISOLATE TIMES FOR LA FAILURES DURING CLIMBING TURN 


LA Failure 

Trigger 

Isolate 

Size 

Time (sec) 

Time (sec) 

100% 

10.9 

11.7 

80 

11.3 

12.15 

60 

11.65 (First) 

16.60 

40 

12.15 (First) 

Verify Fails 

20 

None 

None 


Doublet Maneuvers - Several doublet maneuvers were simulated to test the 
algorithms reaction to severe maneuvers with a broad spectrum of excitation 
frequencies. The doublet maneuvers are defined by the following pitch and 
bank angle commands. 


Pitch 

Doublet : 

At 

~ 10 sec 

e c 

= 5 

deg 



At 

~ 15 sec 

0c 

= -5 

deg 



At 

— 25 sec 

0c 

= 0 

deg 

Roll 

Doublet: 

At 

~ 10 sec 

®c 

= 45 

deg 



At 

~ 15 sec 

0c 

= -45 

deg 



At 

- 25 sec 

0C 

= 0 

deg 


These runs were made with no turbulence and sensor noise was added in a post 
simulation processing function (i.e., not fed back to the commands in the con- 
trol law). Figures 5-50 to 5-55 show the residuals for the roll doublet with 

no noise. 


178 

















ALPHATECH, INC. 

During the pitch maneuver with no failure, several LHT triggers occur, 
but the verify test either runs out of time or indicates a false trigger. 

Figure 5-56 shows the LHT trigger and verify tests for this case. For the 
roll doublet, triggers of every control element occur at one time or another, 
however, failures are not identified because the verify tests either run out 
of time or indicate a false trigger. 

Examination of the residuals during these maneuvers indicates that the LF 
errors are substantially larger than those budgeted in the design. Thus, the 
results are not unexpected. In fact, it is surprising that in spite of the 
many false triggers, no failures are incorrectly declared. Apparently, the 
verify tests perform their function of declaring false triggers quite well 
during these maneuvers. 

Forced Decisions - One of the options which was discussed in subsection 
3.3 regarding the design of sequential tests was the possibility of making a 
fixed sample size decision when a sequential test's time limit is reached. It 
would be expected that smaller failures could be detected in this case, but 
that the likelihood of false alarms during severe maneuvers (e.g. , the doublet 
maneuvers) would increase. Since 100 percent missing elevator failures could 
not be detected during the climbing turn maneuver, we decided to examine the 
results of this case when isolation decisions are "forced" at the time 
limits. 

The forcing of decisions is accomplished by comparing the isolation sta- 
tistic Si(Ni)(see 5.1.3 for definitions) to zero. If it is larger than zero, 
the isolation test "passes" (decides in favor of Hi), and if it is less than 
zero, the test fails (decides in favor of Hj). Figure 5-48 indicates that 
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TIME (SECS) 


R-4466 


Figure 5-56 (a). LHT TRIG 
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Figure 5-56 (b). LHT VERIFY 
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this procedure should work very well in the majority of trigger cases for the 
climbing turn maneuver. Simulation results for this case result in both left 
and right elevator failures being detected and isolated (to the corresponding 
LHT or RHT mode) by t-6.4 seconds. In the case of left elevator failures, the 
LHT/RHT test passes due to the forced decisions and for the right elevator 
failure, the LHT/RHT and RHT/R tests pass due to the forced decisions. 

To see the impact of forced decisions on false alarm performance, the 
roll doublet was executed with no failures. Since many triggers occur, the 
likelihood of making an incorrect decision should be larger when isolate test 
decisions are forced at their time limits. However, no incorrect failure 
identifications are declared for this case. Part of the reason for this per- 
formance is that verify test decisions are not forced. Thus, many false trig- 
gers are indicated due to verify tests being unable to decide at their time 
limits. If verify decisions were also forced, it is possible that false 
alarms could have been declared. 

Threshold Scheduling - Methods for scheduling both fixed sample size 
tests and sequential tests were discussed in subsection 3.3. These methods 
were applied to the aircraft-path FDI system to see if false triggers during 
severe maneuvers could be avoided* 

Basically, threshold scheduling is accomplished through the use of fail- 
ure insensitive estimates of upper bounds on the size of the residuals due to 
errors which were not accounted in the nominal design . In the beginning of 
this subsection, we noted that improved "nominal" performance (in terms of 
detectability levels) could be achieved by reducing the design envelope and 
scheduling thresholds so that periods of potentially large error do not cause 
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incorrect decisions* Thus, a natural choice of scheduling parameters would be 
of the form, 

Ei(k) = l | Kij y-j | (5-71) 

J 


where j is summed over the measurements which were ignored in the error budget 
and K^j represents an upper bound on the error in the corresponding dimen- 
sional coefficient. (Note that since these errors represent errors in the 
residuals, and the residuals are high pass filtered, the "error terms" must 
also be high pass filtered). Unfortunately, the results obtained with such a 
procedure are not adequate. Figure 5-57 shows the LHT trigger test for the 
pitch doublet maneuver with no noise or turbulence. The threshold schedules 
correspond to five percent errors in the coefficients which were ignored in 
the error budget. Clearly Eq. 5-71 does not characterize the errors which are 


being experienced during this maneuver. 



Figure 5-57. 


TIME (SECS) 


R-4468 

LHT Trigger Threshold and Statistic Threshold 
Scheduled as a Function of "Ignored Errors" 
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Several attempts were made to find the proper signals upon which thresh- 
old scheduling could be based. These included the use of deflection measure- 
ments, acceleration measurements, angular rates, and others. No combination 
was found which satisfied the requirement of eliminating false triggers during 
all doublet maneuvers. More investigation of these issues is, therefore, 
necessary. However, we note that in the search for appropriate threshold 

schedules, it was observed that errors in the longitudinal residuals may be 

• • # 

correlated with a. Since no a term was modeled in the residual generation 

equations , it is possible that this is an important source of error. 

Discussion - There are several conclusions about the aircraft-path FDI 
algorithm and its design methodology which can be drawn from these tests. 
Broader conclusions about the entire effort are given in Section 6. The 
conclusions drawn here are based on the results which have been presented as 
well as a more detailed analysis of several simulation results. 

In general, we feel that the aircraft path FDI algorithm performed quite 
well, and as expected , under conditions for which it was designed. It per- 
formed surprisingly well during conditions in which errors were substantially 
larger than those included in the design procedure’s error budget. 

Room for improvement, of course, still exists, and several comments in 
this regard are given below. 

1. Although no engine failure tests were run, examination of throttle 
commands indicates that some engine failure modes (e.g. , stuck: in 
the aircraft path) may produce lower frequency signatures than the 
other failure modes. In order to accommodate such modes, it may be 
desirable to reduce the bandwidth of the HPF only for tests involving 
the engine failures. 

2. Increased sensitivity to failures and greater flexibility in the 
design of isolation tests may be achievable if the full matrix of 
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isolation tests (i.e., "pure" rejection tests) is implemented. This 
is as opposed to the combined test described by Fig. 5-25. 

3. Decreased decision delays may be achievable by starting the sequen- 
tial tests prior to the trigger time. This process would require 
storage and processing of residuals over the "rollback" interval. 

The disadvantage of such an approach is that the same errors which 
might create false triggers could then be present in the verify tests 
(i.e., the verify test is not an independent test). 

4. In keeping with the desire to make the verify test independent to 
reduce false alarms, we could reinitialize the HPFs at the trigger 
time. Although this would remove the effect of large LF errors which 
cause false triggers from the verify test, it would also remove the 
effect of failures. Continued excitation during the sequential test 
interval is then required. This procedure might be most effective 
when a "rollback" interval is used. 

5. The FDI interval and the length of time in which the failure signa- 
tures remain constant are close enough to warrant investigation of 
noncoherent processing schemes. That is, trigger, verify, and iso- 
late tests might perform better if they were based on sums of pro- 
jected and squared residuals. 

6. The sequential test design procedure seems to produce a relatively 
conservative choice of thresholds when errors are within the budget. 
This conclusion is based on observing that, in cases where failures 
are not correctly detected, the isolation tests which run out of time 
are clearly heading in the direction of a correct decision. Some 
further refinement and/or analysis of the current design procedure is 
desirable. 

7. When high frequency unmodeled errors are present, it is possible that 
the likelihood of making incorrect decisions early in a sequential 
test could increase. If adequate high frequency error models are 
used in the design procedure, the thresholds will increase to accom- 
modate this. However, an alternative scheme which might allow 
greater sensitivity in the sequential tests would be to start the 
sequential test with large thresholds and reduce them as the high 
frequency errors become less important. No formal theory or heuris 
tic design procedure exists for such a concept and if important, the 
concept should be investigated further. 

8. More work is needed to define meaningful threshold schedules for 
operation of the algorithm during severe maneuvers. 

9. The moving window average nature of the trigger tests is a very 
inefficient implementation of a low pass filter (LPF). It was chosen 
on the basis of the optimal fixed sample size hypothesis tests. 

Other LPF filter structures which achieve similar noise rejection and 
perhaps have a better transient response with less computation may be 
of interest. 
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5.2 ACTUATOR PATH FDI ALGORITHM DEVELOPMENT 

Two distinct actuator path FDI algorithms were investigated for this 
project. The first algorithm is a "fixed" or constant threshold algorithm 
which maintains a desired false alarm rate during worst case model error at 
the expense of reduced failure sensitivity during all phases of operation. 

The second algorithm uses the threshold scheduling concepts developed in sub- 
section 3.3 to achieve reduced failure sensitivity only during times when 
large error is expected. This latter approach was demonstrated on only two 
surfaces (left and right aileron) because errors other than those assumed in 
this approach were evident in the other surfaces. The development of both 
algorithms and corresponding simulation results are described below. 

5.2.1 Residual Generation 

Both actuator FDI algorithms described in this section are based on the 
same residual generation mechanism. For this project, "open-loop" (see sub- 
section 3.2) actuator residuals are generated by passing the commanded control 
value through an actuator model to produce an estimate of the "actuator" out- 
put position and subtracting this estimate from a measurement of the actuator 
output. Although other residual generation mechanisms are possible (e.g. 
finite memory or closed loop residuals) it is felt that this mechanism pro- 
vides the best ratio between failure sensitivity and model error effects. 

The actuator model (and, hence, what we mean by an actuator) is defined 
by the locations of actuator commands and output measurements. Table 5—13 
shows the definitions used for this project. These definitions assume that 
the only input which drives each control element is computed by a digital 
flight control system (DFCS) that incorporates both stability augmentation and 
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TABLE 5-13. B-737 ACTUATOR DEFINITIONS 


SURFACE 

COMMAND 

OUTPUT 

UNITS 

L&R Stabilizer 

DFCS Command 

Position at Hinge 

degrees 

L&R Elevator 

DFCS Command 

Position at Hinge 

degrees 

Rudder 

DFCS Command 

Position at Hinge 

degrees 

L&R Aileron 

DFCS Command 

Position at Hinge 

degrees 

L&R Throttle 

DFCS Command 

EPR (Scaled) 

k-lbs 


pilot inputs. Thus any failure between the DFCS output and the control mea- 
surements is considered as an actuator failure. Control surface deflection 
measurements are taken on the hinge (possibly using an LVDT on a control rod 
or an RVDT on the hinge itself). 

The estimated control value (for the "next pass") is obtained using a 
generic actuator model which incorporates linear first order dynamics, rate 
and position limits, and static cable stretch. The sequence of operations 
used to generate the control estimate and residual for each actuator are. 

Dynamics : 5k = e -(J) ^6k— 1 + (1~ e w ^) 6 c (k— 1) (5-72a) 

Rate Limiter; IF [ 6k “ 6k-ll/A > RLIM 


Cable Stretch: 


Position Limit: 


Residual : 


THEN S k = 3k-l + sgn(6k “ 5k-l) • RLIM* A 

A 1 A 

<$k = = 6 k 

1 + Q *SF 

IF (6k > Max) THEN 6k = Max 

IF (6 k < Min) THEN 6 k = Min 

Vk = 6]jj(k) - 6k 


(5-72b) 

(5-72c) 


(5-72d) 

(5-72e) 
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where 6 m denotes the control measurement, $ denotes the estimate, Min and Max 
are the position limits, Q is dynamic pressure, SF is a stretch factor, RLIM 
is the rate limit, m is the first order bandwidth and sgn is the signum func- 
tion. The parameters of this model for each control element are given in 
Table 5-14. The sample time. A, is .05 seconds. The elevator command and its 
corresponding measurement is assumed to be given relative to the mechanical 
trim while all other surface positions are relative to some body oriented 
coordinate frame. For the throttle, different bandwidths are used for spool 

A . 

up (6 C > 6k-i) and spool down ( 6 C < 6k-l'* 

TABLE 5-14. ACTUATOR MODEL PARAMETERS 


SURFACE 


0) 

RLIM 

SF 

MIN, 

MAX 

UNITS 

L&R Stabilizer 

1.5 

r/s 

±10 deg/s 

0.0 

-14, 

3 

degrees 

L&R Elevator 

22 

r/s 

±20 deg/ s 

.0023 

4 

"10, 

10 

degrees 

Rudder 

22 

r/s 

±18 deg/s 

0.0 

-10.3 

, 10.3 

degrees 

L&R Aileron 

20 

r/s 

±20 deg/s 

.0016 

-20, 

20 

degrees 

L&R Throttle 

2.0 

r/s (spool up) 

None 

0.0 

(10), 

(60) 

k-lbs 


1.0 r/s (spool down) 


Several error sources which result in non-zero residuals are present. 

The first, and most obvious is sensor errors. These include bias and sensor 
noise as detailed in Table 5-15. Scale factor errors result from errors in 
SF. Errors which are excited by high frequency commands are present due to 
inaccuracies in the dynamic model. Finally, rate limit errors may be present. 
This last error is particularly true for the rudder in which the actual rate 
limit is a nonlinear function of surface position. Also, the throttle command 
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TABLE 5-15. SENSOR ERRORS 


SURFACE 

NOISE (l-o) 

BIAS (MAX) 

UNITS 

L&R Stabilizer 

.1 

.1 

degrees 

L&R Elevator 

.1 

.1 

degrees 

Rudder 

.1 

.1 

degrees 

L&R Aileron 

.1 

.1 

degrees 

L&R Throttle 

(.01) 

(.02) 

k-lbs 


to EPR model may have scale factor errors due to uncertainty in the scaling of 
both commands and EPR to thrust units (klbs). 

Simulation runs using the maneuvers defined in subsection 5.1.4 were made. 
Figures 5-58 to 5-66 show the residuals for each control element during the 
roll doublet maneuver with no noise or failures. In these figures, the initial 
bias is removed. There are clearly many error sources besides sensor noise 
present. For the ailerons and elevators errors occur mainly during the command 
steps which initiate each phase of the maneuver. These errors are consistent 
with dynamic inaccuracies at high frequencies. The stabilizer errors have a 
lower frequency content than those of the elevators and ailerons. These errors 
are also consistent with high frequency dynamic errors since the bandwidth of 
the stabilizer is substantially lower. For the errors observed in the rudder 
and throttle, it is likely that scale factor errors and the existence of vari- 
ous nonlinearities in the simulated engine models play a large role in contrib- 
uting to residual errors. For the rudder, it was discovered that nonlinear 
rate limits are used in the simulation to account for varying aerodynamic loads 
during surface motion. The error between this nonlinear rate limit and the 
constant rate limit used in Eq. 5-72b is evident in Fig. 5-62. 
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Figure 5-59. RT RES 
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Figure 5-60. LS RES 



Figure 5-61. RS RES 
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TIME (SECS) 

R-4475 


Figure 5-64 • RE RES 



TIME (SECS) 


Figure 5-65* AL RES 


R-4476 
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Figure 5-66. RA RES 


These results motivated the development of the two decision processes 
discussed at the beginning of subsection 5.2. The primary focus for this 
project was on test and validation of a fixed threshold algorithm for every 
control element. Time permitted only preliminary investigation of the thresh 
old scheduling algorithm for the aileron controls. 


5.2.2 Fixed Threshold Actuator Decision Process 

A structure for a fixed threshold actuator FDI algorithm is shown in Fig. 
5 - 67 . This structure implements the trigger/verify procedure described in 
Section 3 for dealing with unknown onset times. As in the aircraft-path FDI 
decision process, a "prewhitening" filter which assumes that the primary error 
sources are white noise and very low frequency error is employed. This filter 
is a high pass filter whose cutoff frequency is determined by the length of 


196 




ALPHATECH, INC 


RESIDUAL GENERATION TRIGGER 



Figure 5-67. Fixed Threshold Actuator FDI Structure 
time failure signatures are expected to remain constant, the length of time 
expected to be needed for failure detection (trigger and verify), and the 
bandwidth of the very low frequency errors. We assume that about one second 
is needed for failure detection, that the failure signatures of importance are 
coherent over this interval, and that the very low frequency errors have band- 
widths of less than 0.1 r/s. The high pass filter bandwidth is selected at 
0.5 r/s (i.e., between .1 r/s and 1 r/s). The HPF is implemented with a first 
order digital filter defined by 


1 - 


(l-a)z 1 


(5-73) 


1-az 1 

where a = and where z -1 is a unit delay. 

The trigger process is based on the log— likelihood ratio test for distin- 
guishing a bias of unknown sign in white noise from white noise. The trigger 
statistic is. 
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S T (k) = -jj £ v(k-j+l) (5-74) 

j=l 

The trigger design procedure is as follows. First, N is chosen as 1/ A times 
the desired trigger delay (0.5 second — > N = 10). This choice will ensure 
adequate averaging of high frequency errors while not reducing the effect of 
failures. Next, the threshold is set so that the likelihood of false alarms, 
during worse case errors, is small. Worst case errors, E wc , were determined 
from simulation runs using the three doublet maneuvers, and are shown for each 
control in Table 5-16. If the white noise on each control measurement is 0.2 
degrees (twice the allotted size in Table 5-15 for safety; this does not 
impact the design a great deal), then the thresholds are selected from 

t 0 - E wc + 3o//N" (5-75) 

where a is the .2 degree noise standard deviation. The results are shown in 
Table 5-17. 

TABLE 5-16. WORST CASE ERRORS 


SURFACE 



L&R Stabilizer 

0.5 

deg 

L&R Elevator 

0.75 

deg 

Rudder 

3.0 

deg 

L&R Aileron 

0.50 

deg 

L&R Throttle 

1.5 

klbs 


The verify process is based on the SPRT for the alternate hypotheses 
defining the trigger test. The verify statistic is, 

k k 

S v (k) =| I G V v(j) | - I G c (5-76) 

j=k f j=k c 
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TABLE 5-17. TRIGGER THRESHOLDS 


SURFACE 



L&R Stabilizer 

0.6 

deg 

L&R Elevator 

0.85 

deg 

Rudder 

3.1 

deg 

L&R Aileron 

0.60 

deg 

L&R Throttle 

1.6 

klbs 


The test is run until k = kf + Ny. The time limit Ny is taken to be 10 
samples (0.5 seconds). This is deemed long enough to accomplish averaging of 
high frequency errors and short enough so that failure signatures are not 
substantially reduced. The parameter Gy is chosen as one and G c is one half 
the minimally detectable failure signature, m/2. The minimally detectable 
signature is taken to be twice the worst case error. Finally, the threshold 
is determined so that false triggers are frequently rejected by Ny/2 and 
minimally detectable failures are frequently detected by Ny/2. This is 
accomplished by choosing the threshold as the expected value of Sy(Ny/2). 
Table 5-18 shows the selected thresholds. 

TABLE 5-18. VERIFY THRESHOLDS 


SURFACE 



L&R Stabilizer 

2.5 

deg 

L&R Elevator 

3.75 

deg 

Rudder 

15. 

deg 

L&R Aileron 

2.5 

deg 

L&R Throttle 

7.5 

klbs 
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False alarm checks were made by processing the residuals during several 
maneuvers with no failures according to the decision process described above. 
The three doublet maneuvers and the climbing turn maneuver with noise added to 
the sensors before processing resulted in no false triggers as expected. The 
climbing turn maneuver with turbulence and sensor noise resulted in false 
triggers of the left and right ailerons but the verify processes were able to 
reject these triggers as false in each case. The trigger and verify tests for 

the left aileron are shown in Fig. 5-68. 

Detection checks were then made using the climbing turn maneuver and 
stuck failures (i.e., stuck at the position at the time of failure) for each 
control element. Unfortunately, only runs with turbulence and sensor noise 
were made. Since turbulence tends to excite the controls, the detection 
performance discussed below is presumably better than that which would be 
obtained without turbulence in the simulation. The failure is implemented at 
5.0 seconds and the maneuver occurs at ten seconds. Table 5-19 shows the 
times at which the stuck failures are triggered and verified. The label 
(first) indicates that one or more false triggers were indicated by the verify 
process before a trigger is verified. Throttle failures are not detected 
because the deviation of the measured EPR from the estimated EPR (in klbs) 
does not exceed the minimally detectable level. Only one trigger occurs for 
the left throttle, but the signature is not large enough to be verified. This 
is shown in Fig. 5-69. The right elevator failure is of some interest since 
several triggers occur before the failure is verified. This is shown in Fig. 
5-70. For the first trigger the signature is so small that the verify SPRT 
passes its negative threshold. For the second trigger, the verify time limit 
is reached and the failure is identified on the third trigger. 
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TABLE 5-19. TRIGGER AND VERIFY TIMES FOR STUCK CONTROLS 
SURFACE TRIGGER TIME (SEC) VERIFY TIME (SEC) 


Left Throttle 

24.6 


Right Throttle 

None 

None 

Left Stabilizer 

11.25 

11.5 

Right Stabilizer 

10.9 

11.1 

Rudder 

6.15 

6.3 

Left Elevator 

6.05 

6.3 

Right Elevator 

7.0 (first) 

10.7 

Left Aileron 

5.45 (first) 

6.4 

Right Aileron 

5.95 

6.1 


The fact that throttle failures were not detected must be weighed against 
the severity of the failure. Inspection of the aircraft response for both 
failed and unfailed cases shows that almost no performance is lost during this 
maneuver when a single throttle is stuck. The difference appears in the air- 
speed in which the lowest airspeed achieved during the maneuver is five knots 
lower with the throttle failure. Otherwise the force and moment imbalances 
resulting from the throttle failure seem to be adequately compensated by the 
nominal FCS of [58]. 

DISCUSSION 

The simulation results indicate that the actuator path FDI process 
described by Fig. 5-67 can be designed to virtually eliminate false alarms 
during realistic and severe maneuvers. However, the design relied on observa- 
tion of the size of normal errors. Estimation or bounding of such errors is 
not an easy task for real aircraft. Thus, extensive flight records may need 


204 



ALPHATECH, INC. 


to be examined to determine the size of errors. If these errors are expected 
to be constant over the flight envelope and over time, and if they are small 
enough (as in this study) so that important failures can be detected, then the 
syst6iu described above can be made to perform adequately. Otherwise, schedul 
ing and/or adaptation mechanisms may be required. 

5.2.3 Scheduled Threshold Actuator FDI Decision Process 

One of the limitations of the fixed threshold algorithm is that the 
thresholds must take into account worst case error. This can sometimes limit 
the sensitivity to failures during times when this error is not likely to 
appear. The decision process developed here utilizes the single-input, single- 
output threshold scheduling ideas developed in subsection 3.3 to adjust thresh- 
olds when errors (due to dynamic model errors) are expected to be large. Time 
permitted only application to the aileron actuator system. These were chosen 
since the residuals suggested that dynamic errors were of greatest importance 
for these actuators. 

The structure for the alternate actuator decision process is shown in 
pig. 5-71. The trigger is based on the development in subsection 3.3. It is 
assumed that an error filter exists whose squared magnitude bounds the (rela- 
tive) frequency dependent errors in the residuals. That is, the error filter 
in Fig. 5-70, Zm(w) , must bound E(w)/H(w) where E(u>) is defined in Eq. 3-125, 
and H/oj) is the true actuator transfer function. The trigger equations are, 

SrOO-i l v(k-j + l)2 (5-77) 

1 j-1 

i 

t T (k)=to+- l e(k-j + l)2 (5-78) 

1 j=l 
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R-3793B 


Figure 5-71. Scheduled Threshold Actuator FDI Structure 

LPF « (Low Pass Filter); D T = (Constant Multiplier) 

The trigger window length, N^, must be chosen as long as possible to ensure 
that the impact of the approximation of Eq. 3-128 by a finite sum is not too 
severe, but should not be longer than the desired FDI delay if possible. 
Since the time constant of the aileron actuator model is small, N«p is chosen 
as 10 (0.5 sec.). This will permit substantial averaging of the high fre- 
quency content in the residuals. The error filter is a first order HPF with 
break frequency corresponding to the aileron actuator model bandwidth. This 


206 



ALPHATECH, INC 


is based on the assumption that the first order actuator model is inaccurate 
above its break frequency. The nominal threshold, t Q , is now selected only to 
account for sensor noise since we assume that all model error will be 
accounted for by the threshold schedules. Thus, if v(k) is white noise with 
intensity of .1 degrees, then the variance of S-p is about .02 degrees. We 
choose t Q as three times this or .06 degrees. Notice that this is substan- 
tially smaller than the fixed threshold algorithm. Thus, very small actuator 
failures can be detected during periods of quiescent operation (low frequency 
actuator commands) in which the threshold offsets will be small. 

The verify process is based on the assumption that the nonconstant part 
of Eq. 5-78 is an estimate of the size of the residual error (due to dynamic 
mismatches) at time k. This estimate is used to modify the verify thresholds 
to maintain the likelihoods of error at each stage of the process as described 
in subsection 3.3. 

The constant is used to modify the high frequency gain of the error 
filter. The verify statistic is defined by the constants Gy and G c . As in 
the trigger process Gy is chosen to be one and G c is one half of the minimally 
detectable failure signature. Without errors, this signature is quite small. 
We chose, however, to use 0.5 degrees as a minimally desirable failure signa- 
ture since smaller signatures are deemed to be unimportant and since other 
errors besides those accounted for by threshold scheduling may be present. 

The nominal threshold is chosen as in subsection 5.2.1 and takes the value of 
1.25. 

Several experimental design iterations were made to determine a suitable 
value for Kfl. Choosing so that the error filter has a high frequency gain 
of 1 (i.e., 100% error relative to the surface estimate) resulted in no 
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triggers during doublet and climbing turn maneuvers with no turbulence and 
post simulation added sensor noise. False triggers (no false alarms) occurred 
during the climbing turn with turbulence. Figure 5-72 shows the aileron trig- 
ger for the climbing turn maneuver. The schedules seem to be conservative 
(too large) but occur at the proper times. Figure 5-73 shows the same test 
when turbulence is added. The large high frequency excitation of the control 
surface results in errors which appear to be larger than those accounted for 
with K H chosen to make the high frequency gain of the error filter equal to 1. 
It was determined that the high frequency gain of the error filter needed to 
be ten (k H - 7) to prevent triggers during the climbing turn with turbulence. 
Figure 5-74 shows the resulting trigger test. The threshold offset for this 
test is frequently larger than required. Although it is believed that large 
relative error is indeed possible for some frequencies, it is likely that it 
is not constant over a large frequency range as in the error filter. This may 
explain the over conservativeness of the test shown in Fig. 5-74. An alter- 
nate method which utilizes absolute error can easily be derived (amounting to 
using the actuation command instead of the estimate in the error filter) and 
may produce better results* 

Finally, detection checks were made using simulations of stuck actuators 
during the climbing turn maneuver with turbulence and sensor noise. Although 
failures are easily detected, no significant improvement in detection time 
(over the fixed threshold case) was observed. Figure 5-75 shows the trigger 
and verify tests for this case. 

DISCUSSION 

It is beieved that more work on the threshold scheduling method is needed 
before meaningful conclusions can be drawn. 
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Figure 5-74. LA TRIG (KH = 7) ( = Statistic) (— — = Threshold) 
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Figure 5-75a. RA TRIG ( = Statistic) • = Threshold) 
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Figure 5-75b. RA VERIFY ( = Statistic) (-•— = Threshold) 


21 





ALPHATECH, INC. 


SECTION 6 

SUMMARY OF RESULTS AND CONCLUSIONS 

This effort has developed and explored the use of a decentralized 
approach to failure detection and isolation for use in restructurable control 
systems. This work has produced, 

1. A method for evaluating fundamental limits to FDI performance, 

2. Application of this method using flight recorded data, 

3. A working control element FDI system with maximal sensitivity to 
critical control element failures, 

4. Extensive testing of this system on realistic simulations, and 

5. A detailed design methodology for this system which involves 
parameter optimization (with respect to model uncertainties) and 
sensitivity analyses. 

For this project, we have concentrated on detection and isolation of generic 
control element failures since these failures frequently lead to emergency 
conditions and since knowledge of remaining control authority is essential for 
control system redesign. The failures we considered are generic in the sense 
that no temporal failure signature information was assumed. Thus, various 
forms of "functional failures" are treated in a unified fashion. Such a 
treatment results in a robust FDI system (i.e. , one that covers all failure 
modes) but sacrifices some performance when detailed failure signature infor 
mation is known, useful, and employed properly. We assumed throughout this 
project that all sensors are validated (i.e., contain only in-spec errors) and 
that only the first failure of a single control element needs to be detected 
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and isolated. The FDI system which has been developed will handle a class of 
multiple failures. 

The FDI system which was developed using the design methodologies out- 
lined in Section 3 worked quite well during simulation tests on data from 
NASA’s modified B-737 simulation. This is true despite large errors between 
the models used in the FDI system and the models used in the simulation. Fur- 
thermore, this system worked as predicted when errors between the simulation 
model and the FDI models were within the envelope used to choose FDI parame- 
ters (thresholds, etc.). The design methodology discovered the inherent 
indistinguishability of same-side elevator and stabilator panels (on the basis 
of force and moment imbalances alone) early in the project resulting in a sys- 
tem which does not attempt to do what is impossible. 

Improvements in the design methodology, system detail, and implementation 
are still possible. Some over-conservatism was observed in the design proce- 
dures for sequential tests. Alternative filter structures may speed up the 
FDI process. The use of "probe" signals during the isolation phase would 
allow enhanced isolation capability, (including the isolation of stabilizers 
from elevators). And, finally, further exploration of the importance and 
design of threshold schedules is needed. 

The remainder of this section provides a more complete summary of this 
report. It also provides a discussion of some of the unexplored alternatives 
in the design, implementation, and testing of the FDI system, and suggestions 
for further work. 

DECENTRALIZED FDI 

In Section 3 of this report, ALPHATECH T s decentralized approach to FDI 
was described. This approach requires an assessment of all sources of 
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redundancy (including analytical redundancy) and the utilization of this 
redundancy to produce decoupled or decentralized residual signals that are 
then processed to produce FDI decisions. This method represents a suboptimal 
approach to the data fusion problem (i.e., combining all sources of informa- 
for decisionmaking purposes) under ideal circumstances (no modeling 
error), but prevents the mixing of well known relationships with poorly known 
relationships when modeling error exists. As a result, the decentralized 
approach can be superior when significant modeling errors exist. 

The first application of this idea to the control element FDI problem 
resulted in a decomposition into actuator-path and aircraft-path subproblems. 
Measurements of actuator inputs and outputs allowed this decomposition. The 
advantage of this decomposition is that uncertainties about aerodynamic models 
do not affect the ability to detect actuator path failures and uncertainties 
about the actuation mechanism do not affect the detectability - of aircraft path 
failures. The disadvantage of this decomposition is that some configurations 
of actuator output sensors may be vulnerable to failure thereby complicating 
the (assumed) sensor validation process. Fortunately, however, the distinc- 
tion between the aircraft-path and the actuator-path is somewhat arbitrary so 
that the aircraft path subsystem can easily be designed to detect actuator 
failures when actuation outputs are not available. 

The second application of decentralization is in the method of forming 
aircraft-path residuals. Since only force and moment balance relationships 
are affected by aircraft-path failures, residuals based on these relationships 
are desired. Translational accelerometer measurements allows this to be 
accomplished for the force balances, however, rotational dynamic relationships 
must also be used since rotational acceleration is not measured. This method 
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of forming aircraft path residuals has two advantages. The first is that 
errors in other relationships which would affect centralized (Kalman filter- 
like) residual generators do not affect these decentralized residuals. A 
prime example is the effect of acceleration of the air mass on the transla- 
tional kinematic relations. The second advantage is that control element 
failures show up in fixed directions in residual space. This permits the dis- 
tinguishability of aircraft-path control element failures independent of the 
detailed failure signatures. Only the relative magnitude of imbalances in 
different residuals is necessary for failure isolation. 

FDI SYSTEM 

The structure of a generic FDI system which efficiently solves the 
unknown onset time problem was developed in subsection 3.1. This structure 
involved a monitoring or trigger process which is used to reject the hypothe- 
sis of normal operation and to trigger a verification and isolation process to 
reject false triggers and to identify the source of a failure. This structure 
is used to achieve performance advantages which approach the performance of 
the known onset-time case. These advantages include greater failure sensitiv- 
ity, lower false alarm rates, and shorter detection delays. 

Aircraft-Path Subsystem - The aircraft-path trigger was designed to make 
the probability of missing a critical failure small. Thus, each failure mode 
had an explicit trigger function which is optimized for triggering under the 
corresponding failure mode. Each trigger satisfies the condition that IF a 
particular minimal failure occurs, THEN the corresponding trigger test will 
"pass." Since the converse is not true and since false triggers are possible, 
we need to perform verify and isolate tests. 
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The verify and isolate tests are sequential tests and are designed so 
that failures which are larger than some minimal value will be detected and 
isolated in shorter time periods. If they reach a maximal time limit, no 
decision is made, although it is also possible to make a fixed sample size 
decision at this time. 

The isolation process recognizes the fact that only the rejection of 
failure modes is possible when signature information is not used. This fact 
results in principle, in a matrix of isolation tests, each designed to reject 
a failure mode with maximal sensitivity to another failure mode. Although 
this structure appears complex, it guarantees optimal performance for every 
failure mode and allows detailed analysis and optimization of each part of the 
system. In practice, the off-diagonal tests in this isolation matrix were 
combined for efficiency. Also, in principle, only those failure modes which 
are in the "trigger-implied ambiguity group" need to be isolated, although in 
practice all failures were consdiered as possible following any trigger. To 
declare a failure, all isolation tests must "vote" in favor of that failure. 

Actuator-Path Subsystem - The character of the actuator residuals (all actua- 
tor failure directions are mutually orthogonal) resulted in one actuator— path 
subsystem for each actuator failure. Thus, no isolation process was needed. 
These subsystems, like the aircraft-path subsystem, also used a trigger/verif y 
structure to "solve" the unknown onset time problem. Two decision processes 
were created and tested; a fixed threshold and a varying threshold algorithm. 

The fixed threshold algorithm was designed to accommodate the observed 
low frequency behavior in each residual, sensor noise, and other high frequency 
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errors. The result of a trigger crossing its threshold is the initiation of a 
sequential verify test. If the verify test passes, the corresponding control 
element is declared as failed. If a verify fails, a "false trigger" is 
declared. Because fixed thresholds were used to accommodate low frequency 
errors, the sensitivity to actuator path failures was higher than originally 
expected (though by no means unacceptable). 

The varying threshold algorithm was based on the concept derived in 
Section 3 for single-input, single-output systems with transfer function 
errors. It assumed that all transfer function errors were high frequency 
relative errors. Observations clearly indicated that this was not the case, 
and consequently, this decision process did not perform as well as expected. 
Further work in this area is needed before substantive conclusions can be 
drawn. 

DESIGN METHODOLOGIES 

One key to the successful deployment of any FDI concept (or any aircraft 
system concept for that matter) is the development of an analytical design 
methodology which permits engineers to assess the impact of various contin- 
gencies on performance and modify the design accordingly. In Section 3 of 
this report, we have outlined the structure of such a methodology and have 
given examples of how the many analysis and synthesis tasks could be per- 
formed. The reason such a methodology is important is that exhaustive simula- 
tion and flight testing for purposes of design is expensive and may never 
encompass all contingencies of interest. An analytical design method provides 
quicker and cheaper answers to the same questions and should allow all contin- 
gencies to be considered. 
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For the FDI problem, the contingencies of interest are the size of poten- 
tial error sources and the magnitude and character of the maneuvers which 
excite them. In Section 4 we demonstrated how an error budget might be formed 
and utilized in the design process. This error budget serves to define a 
truth model for which parameters in the FDI system (which is designed on the 
basis of a design model) are optimized. 


SIMULATION RESULTS 

Substantial testing of both subsystems was performed using data from 
NASA's B-737 simulation. These tests included drastic model mismatches 
between the FDI models and the actual simulation model, sensor noise and scale 
factors and turbulence. Various combinations of sensor noise and turbulence 
were used to evaluate the impact of each on the FDI system. A total of 41 
60-second simulation runs were made by NASA and data recorded for use in this 
project. All runs were made using the control law of [58]. 


Aircraft-Path Subsystem - Three categories of simulation runs were used 
to test this subsystem. These included: 

1. False alarm checks with three "doublet" maneuvers and a climbing 
turn. The climbing turn checks were made with no turbulence or sen- 
sor noise, with sensor noise alone, and with both sensor noise and 
turbulence. The doublet maneuver checks were made with no sensor 
noise and no turbulence and with sensor noise and no turbulence. 

2. Detection checks were made using 100 percent effectiveness failures 
of all control elements except the engine (simulation capability for 
this case was not available) during a climbing turn maneuver. These 
checks were made with sensor noise alone — no turbulence. 

3. Detection checks were made for varying levels of partial effective- 
ness failures for the left aileron during a climbing turn with sensor 
noise and no turbulence. 
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For the nine false alarm check runs which were made, no false alarm was 
ever generated. No false triggers occurred during any of the three climbing 
turn cases. During the two pitch doublet maneuvers triggers of the left "hor- 
izontal tail" control occur, but are verified as false. During the two roll 
doublet maneuvers, triggers of every control element occur, but are also veri- 
fied as false. 

Of the seven control elements which were failed 100 percent in the second 
category of simulation runs, five were correctly isolated within two seconds 
of either the failure time (for surfaces with loads in straight and level 
flight) or the maneuver time. Left and right elevator failures continually 
caused triggers to occur, but no unanimous decision could ever be reached dur- 
ing the climbing turn maneuver. Fortunately (and as expected), the degrada- 
tion in maneuver performance due to these failures is slight and they clearly 
do not represent an emergency situation for the aircraft with the control law 
being used. In no cases was an incorrect isolation ever made. Of course, no 
attempt was made to isolate same side elevator and stabilizer controls from 
each other since they were deemed indistinguishable (using force and moment 
imbalances alone) during the design process. Thus correct isolation for the 
stabilizers implies isolation to a fictitious horizontal tail surface. The 
elevator failures which were not unanimously isolated turned out to have 

signatures which were considered only marginally detectable by the design 
/ 

methodology and thus this is an expected result. 

Finally, of the five left aileron failures of varying partial effective- 
ness, three were correctly isolated. It was concluded that failures of greater 
magnitude than sixty percent loss could be correctly isolated during this 
maneuver and of less magnitude than forty percent could not be unanimously 
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isolated. The forty percent failure case caused triggers to occur and the 
twenty percent case did not. Isolation times varied from less than one 
second for 100 percent failures to five seconds (two triggers) for the sixty 
percent failure. These results are consistent with the notion that for a 
given maneuver, the signature magnitude decreases with failure severity, thus 
reducing the ability to detect and isolate and increasing the length of time 
needed for the sequential decision process to conclude. 

No detection checks were made with turbulence, however, it would be 
anticipated that due to increased excitation of controls, performance would be 
equal or better to what was observed. Some experimentation with forced iso 
late decisions" were made leading to the conclusion that, for the case tried, 
such a procedure may provide significantly enhanced detection performance with 
little degradation in false alarm performance. More discussion of the results 
is provided at the end of subsection 5.1. 

Actuator-Path Subsystem - For the fixed threshold decision algorithm 
false alarm and detection of stuck-at-f ailure checks were made. The false 
alarm checks were made using a climbing turn with sensor noise and no turbu- 
lence and with both noise and turbulence and using three doublet maneuvers 
with sensor noise and no turbulence. Detection checks were made using the 
climbing turn with sensor noise and turbulence. Time did not permit examining 
the effects of sensor noise and turbulence independently. Thus detection per- 
formance as sited here is presumably better than what would be ovserved with 
no turbulence. 

None of the four maneuvers with noise and no turbulence produced a false 
trigger. When turbulence was added during the climbing turn maneuver false 


220 


ALPHATECH, INC. 


triggers of the left and right aileron were observed, but no false alarms were 
generated. 

Of the nine stuck-at failures implemented, seven were correctly detected. 
Only the engine failures were missed due to the lack of sufficient excitation. 
Examination of the impact of missing these failures showed that little degra- 
dation in performance occurred (five knot transient airspeed difference). 
Failure detection times varied from 150 milliseconds to greater than three 
seconds. The longer detection times were due to the fact that triggers 
occurred in unloaded surfaces before the maneuver as a result of excitation 
due to turbulence. 

The varying threshold algorithm was also tested, however, too few results 
are available to draw significant conclusions. 


UNEXPLORED CONCEPTS, FURTHER WORK, AND OTHER NOTES 

A list of other comments including concepts and implementations which 
were left unexplored, some suggestions for possible algorithm improvement, and 
other notes is provided below. This is in addition to the suggestions made at 
the end of subsections 5.1 and 5.2. 

1. In the design methodology, a more detailed truth model might be used 
to alleviate some of the conservatism inherent in the current design. 
This might include modeling of the pilot and the control law to 
obtain more accurate statistical descriptions of the residuals. Per- 
haps a continuous time domain methodology would be more useful than 
the discrete models currently being used. Note, however, that these 
statistical models would still assume, at least, piecewise station- 
arity and that nonstationarity is a large issue which must be handled 
in some fashion. 

2. In the design methodology, more detailed sensitivity analyses would 
be useful in demonstrating the tradeoffs associated with the design. 

3. Computational savings and even performance improvement may be 
achieved by implementing the verify and isolate tests on a "trigger 
implied ambiguity basis." The ambiguity, of course, depends not only 
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on the physics of the problem, but also on the trigger design* . For 
example, if only the yaw residual is used to trigger rudder failures, 
then a rudder trigger alone eliminates those controls which can not 
6xcit6 ysw imbalances * 


In the design process the notion of an FDI bandwidth arose. Failures 
whose signatures lie outside this bandwidth will not be detectable. 

If it were desirable to detect such failures (especially very low 
frequency signatures), one might consider the design of a multiple 
bandwidth FDI system. Such a system would have higher detectabili 
ties for failures which are outside the current bandwidth without 
sacrificing the performance obtained with the current system. 0 
course, the limiting case of a multiple bandwidth system would be a 
concept involving Fourier transforms and statistics and thresholds 
for combinations of frequency elements. 


5. In the evaluation of fundamental limits to FDI performance, it would 
be useful to process more segments of flight data and to average t e 
results. Use of other spectral estimation methods might also be 
useful, particularly the periodogram approach (i.e., taking averages 
of DFT's). These may be explored in subsequent work. 


6. One dissapointment of the current work was the need to rely on 

observed data in the design process. Studies relating to the crea- 
tion of more accurate error budgets need to be accomplished in order 
for the design methodology to be maximally useful. 


7. A. reduction in computational requirements for the aircraft path algo 
rithm might be achieved by a system which performed only a single 
rejection test for each control element. Such a system would, of 
course, sacrifice sensitivity to failures. However, this approach 
might be tried first and evaluated to see where specific sensitivi 
ties need to be enhanced. The system concepts developed in this work 
could then be applied only where increased sensitivity was needed. 


8. Relaxation of sensor validity assumptions is an important future 
effort. To detect sensor and control element failures, additional 
redundancy relations are needed. The work of Deckert, et al., 116J 
provides an excellent starting point since it was used as a basis for 
the approach developed in this work. Reliability issues also need to 
be addressed in this context to ensure that the tradeoff between 
hardware and analytic redundancy is made properly. 

9. The actuator models used in this work required a substantial amount 
of detail (scale factors, rate limits, etc.). The most significant 
detail seems to have been rate limits since errors due to incorrect 
rate limits can affect both low and high frequency behavior. 


10. For the varying threshold actuator decision process, improve 

performance may be possible if bounds on the absolute error (rather 
than the relative error) are used to schedule thresholds. 
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APPENDIX A 

DERIVATION OF AERODYNAMIC COEFFICIENTS FROM LINEAR MODELS 

Assume that the linear perturbation model was derived through a first 
order Taylor expansion of Euler’s equations. That is, the linear model 

x = Ax + Bu (A-l) 

represents the dynamic perturbations from nominal values xq and uq, derived by 
a linearization of the nonlinear rigid body equations. The state vector x is 
(U, V, W, P, Q, R) and the control vector u is composed of the "deflections," 
Si (including throttle). Equation A-l was derived from a linearization of the 
following equations. 


m(U 

+ WQ • 

- RV) = X 
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The forces and moments (X Y Z L M N) were expressed in Eqs. 5-7 through 
5-18 in terms of a measurement vector, y T = (V x , a, 8, P, Q, R) and the 
nondimensional coefficient which we wish to solve for. Thus, Eqs. A-2 to A- 7 
can be rewritten. 


x = f(y, 5 ; C) 


(A-8) 


jjhere q is a vector of nondimensional coefficients. Since y is a function of 


x the partial derivatives 


8x7 

3xi 


Xr,U 


and 


o u o 


3x£ 

36j | xuU Q 


can be obtained as a func— 


3xi 3xi 3yj 

tion of C (Note: ■ )• These partials are then assigned to the 

3xj 3yj 3xj 

numerical values specified by the A & B matrices in Eq. A— 1 and the coeffi- 


cients C found. 

In particular, if we assume that V x is constant, then the solution for C 
is unique and is given below. 


Stability Terms : 

• * 

c Xa = ( -V T sin « cos + + V T cos « cos 

C XQ = m <|^ + w > 2V T / c QS) 

# • • 

C Y g = [(|J + R) C-V T cos a sin g) + (||) V x cos 8 + ^ “ p > 

(-V x sin a sin g)] / QS 
Cyp =(fp - W) (2 V t / bQS) 

3V — 

C Y r + u) (2V X / bQS) 


(A-9) 

( A-10) 


(A-ll) 

(A-12) 


(A-13) 
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c Za = [ ("|jj “ Q) (“Vx sin a cos g) + (-^-) V T cos a cos g] / QS (A-14) 

C ZQ = m(|| " U) (2V X / cQS) (A- 15) 

• • • 

3P 

C L$ = I x[('^j) < _V T cos « sin + ('gy) V T cos B + ^ 



(~V T sin a sin g) ] / QSb 




(A-16) 

C LP = 

1 * ( i!> 

(2Vx / b 2 QS) 




(A-17) 

C LR = 

u*4> 

+ (I z - I y ) Q] (2 V t 

/ b2Qs) 


(A-18) 

c Ma = 


(-Vx sin a cos g) + 

(J2) 

K 3W' 

V<p cos 

a cos g] / QSc 

(A-19) 

Cmq = 

I y ( 3Q ) 

(2V t / "c 2 QS) 




(A-20) 

c Ng = 

*.[<#> 

(-Vx cos a sin g) + 

( JR) 

v 3V' 

Vx cos 

* + 4> 



(-Vx sin a sin g)] / QSb 




(A-21) 

C N p = 

[^4> 

+ (I y - I x ) Q] (2 V t 

/ b2QS) 


(A-22) 

C NR = 

[^4> 

(2V t / b 2 QS) 




( A-23) 


Control Terms: 
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<=L 61 - Ixtffj) / Q sb 

c M6i = I y ( 36 i ) 1 ^ Sc 
C N6i = 1 Q Sb 


(A-27 ) 
( A-28) 
(A-29) 


In the above, y and u are evaluated at their nominal values and the partials 
are obtained from the values of matrices in the linear model. This method 
ignores contributions to forces and moments due to o and assumes that V-p is 
constant. Relaxation of these assumptions is possible, but the solution pro- 
cedure is more difficult. 

Two flight conditions were evaluated using this method. The constant 
coefficients (basic lift, drag, etc.) are determined by ensuring that x = 0. 
Notice that the nondimens ional coefficients at the two flight conditions are 
substantially different. This is most likely due to large control nonlinear! 
ties and the effect of flap deflection on basic aircraft characteristics and 
on horizontal tail nonlinearities. 


FC1 is defined by: 

V = 160 KIAS 
h = 3500 feet 
G = 0 (Gear Up) 
Flaps = 15° 
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FC2 is defined by: 

V - 140 KIAS 
h = 1500 feet 
G = 1 (Gear Down) 

Flaps = 30° 

Other assumptions 

S = wing area = 1000 ft 2 
b = wing span = 100 ft 
c “ average chord = 10 ft 
m = 

The resulting coefficients are shown below in Table A-l. 
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TABLE A-l. 

FC1 


CXB 

0.1 1 159E+01 ALPH 
0.33830E-01 Q 
0. 1 1541E-01 THRUST_L 
0. 1 1541E-01 THRUST_R 
0.7 1224E-03 D_STAB_L 
0.71224E-03 D_STAB_R 
O.OOOOOE+OO D_RUDDER 
0.33401E-03 D_ELEV_L 
0.33401E-03 D_ELEV_R 
0.38211E-03 D_AILE_L 
0.38211E-03 D_AILE_R 
-.61064E-03 D_SPLR_L 
-.61064E-03 D_SPLR_R 
-.91344E-01 K 

CYB 

-.12764E+01 BETA 
0. 14274E+00 P 
0.40748E+00 R 
O.OOOOOE+OO THRUST_L 
O.OOOOOE+OO THRUST_R 
O.OOOOOE+OO D_STAB_L 
O.OOOOOE+OO D_STAB_R 
0.68085E-02 D_RUDDER 
O.OOOOOE+OO D_ELEV_L 
O.OOOOOE+OO D_ELEV_R 
0.60524E-04 D_AILE_L 
-.60524E-04 D_AILE_R 
0.62347E-03 D_SPLR_L 
-.62347E-03 D_SPLR_R 
-.54122E-15 K 

CZB 

-.69370E+01 ALPH 
-.38821E+00 Q 
-. 16000E-04 THRUST_L 
-. 16000E-04 THRUST_R 
-.83883E-02 D_STAB_L 
-.83883E-02 D_STAB_R 
-O.OOOOE+OO D_RUDDER 
-.39325E-02 D_ELEV_L 
-.39325E-02 D_ELEV_R 
-.44963E-02 D_AILE_L 
-.44963E-02 D_AILE_R 
0.44794E-02 D_SPLR_L 
0.44794E-02 D_SPLR_R 
-.52182E+00 K 


NONDIMENSIONAL COEFFICIENTS 

FC2 


CXB 

0. 12661E+01 ALPH 
0.26276E-01 Q 
0. 15070E-01 THRUST_L 
0. 15070E-01 THRUST_R 
0.52730E-03 D_STAB_L 
0.52730E-03 D_STAB_R 
O.OOOOOE+OO D_RUDDER 
0.25056E-03 D_ELEV_L 
0.25056E-03 D_ELEV_R 
0.28252E-03 D_AILE_L 
0.28252E-03 D_AILE_R 
-.70234E-03 D_SPLR_L 
-.70234E-03 D_SPLR_R 
-. 16288E+00 K 

CYB 

-. 13282E+01 BETA 
0.27300E+00 P 
0.30631E+00 R 
O.OOOOOE+OO THRUST_L 
O.OOOOOE+OO THRUST_R 
O.OOOOOE+OO D_STAB_L 
O.OOOOOE+OO D_STAB_R 
0.68867E-02 D_RUDDER 
O.OOOOOE+OO D_ELEV_L 
O.OOOOOE+OO D_ELEV_R 
0.44354E-04 D_AILE_L 
-.44354E-04 D_AILE_R 
0.85356E-03 D_SPLR_L 
-.85356E-03 D_SPLR_R 
-.55180E-15 K 

CZB 

-.74416E+01 ALPH 
-.42188E+00 Q 
-. 16445E-04 THRUST_L 
-. 16445E-04 THRU ST_R 
-.84034E-02 D_STAB_L 
-.84034E-02 D_STAB_R 
O.OOOOOE+OO D_RUDDER 
-.39927E-02 D_ELEV_L 
-.39927E-02 D_ELEV_R 
— .45015E-02 D_AILE_L 
-.45015E-02 D_AILE_R 
0.60711E-02 D_SPLR_L 
0.607 1 IE-02 D_SPLR_R 
-.10091E+01 K 
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24436E+00 BETA 
-.58055E+00 P 
0.20669E+00 R 
0. 1087 IE-03 THRUST_L 
1087 IE-03 THRUST_R 
0.61458E-03 D_STAB_L 
-.61458E-03 D_STAB_R 
0.75713E-03 D_RUDDER 
0.28741E-03 D_ELEV_L 
-.28741E-03 D_ELEV_R 
0.61045E-03 D_AILE_L 
-.61045E-03 D_AILE_R 
-.70755E-03 D_SPLR_L 
0. 70755E-0E D_SPLR_R 
107 16E-15 K 


-.27875E+00 BETA 
-.57419E+00 P 
0.25012E+00 R 
0.14199E-03 THRUSTJL 
14199E-03 THRUSTJR 
0.61540E-03 D_STAB_L 
-.61540E-03 D_STAB_R 
0.76630E-03 D_RUDDER 
0.29215E-03 D_ELEV_L 
-.29215E-03 D_ELEV-R 
0.63462E-03 D_AILE_L 
-.63462E-03 D_AILE_R 
-.96394E-03 D_SPLR_L 
0.96394E-03 D_SPLR_R 
10434E-15 K 


CMB 


CMB 


-.17373E+01 ALPH 
-.33986E+02 Q 
0. 57743E-02 THRUST_L 
0. 57743E-02 THRUST_R 
-.33534E-01 D_STAB_L 
-.33534E-01 D_STAB_R 
0.00000E+00 D_RUDDER 
15602E-01 D_ELEV_L 
15602E-01 D_ELEV_R 
-.44041E-02 D_AILE_L 
-.44041E-02 D_AILE_R 
0. 18332E-02 D_SPLR_L 
0. 18332E-02 D_SPLR_R 
-.90779E-02 K 


-.24619E+01 ALPH 
-.33929E+02 Q 
0.75395E-02 THRUST_L 
0.75395E-02 THRUST_R 
-.33518E-01 D_STAB_L 
-.33518E-01 D_STAB_R 
O.OOOOOE+OO D_RUDDER 
15877E-01 D_ELEV_L 
15877E-01 D_ELEV_R 
-.43994E-02 D_AILE_L 
-.43994E-02 D_AILE_R 
0.23797E-02 D_SPLR_L 
0.23797E-02 D_SPLR_R 
12474E+00 K 


CNB 


CNB 


0. 13662E+00 BETA 
13149E+00 P 
147 15E+00 R 
0. 18818E-02 THRUST_L 
18818E-02 THRUST_R 
0.13354E-03 D_STAB_L 
13354E-03 D_STAB_R 
26346E-02 D_RUDDER 
0.76452E-04 D_ELEV_L 
-.76452E-04 D_ELEV_R 
0. 14500E-03 D_AILE_L 
14500E-03 D_AILE_R 
35808E-03 D_SPLR_L 
0. 35808E-03 D_SPLR_R 
0.17944E-15 K 


0.14165E+00 BETA 
13610E+00 P 
-.15230E+00 R 
0.24579E-02 THRUST_L 
-.24579E-02 THRUST_R 
0. 12025E-03 D_STAB_L 
12025E-03 D__STAB_R 
-.26849E-02 D_RUDDER 
-.70973E-04 D_ELEV_L 
0.70973E-04 D_ELEV_R 
0. 15738E-03 D_AILE_L 
15758E-03 D_AILE_R 
-.45348E-03 D_SPLR_L 
0.45348E-03 D_SPLR_R 
0.20830E-15 K 
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APPENDIX B 

DERIVATION OF OVERALL ERROR RATE EXPRESSIONS 


This appendix derives expressions for overall system error probabilities 
for the aircraft path FDI system. These probabilities are based on the indi- 
vidual error probabilities of the hypothesis tests which are performed. First 
we define some fundamental events. 

Let, 

event of the i^ trigger indicating possible failure 

event of the i c ^ verify test choosing Hi over H 0 

event of the i, j th isolation test choosing % over 
H-j 

1, . .., 7 (i.e., seven control element failures) 


Ti = 
Vi = 

*i/j = 
i.j. = 


(B-I) 

(B-2) 

(B-3) 


Also, we define the aggregate events. 


7 

T = U Ti = the event that some trigger indicates 

i=l failure (B-4) 


7 

V = y Vi = the event that some verify chooses Hi 

i=l over H 0 (B-5) 


di = TO Vi f| Ii/j = the event that failure i is 

j*i declared (B-6) 


The relevant probabilities which we wish to calculate are. 
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PpA “ probability of choosing any failure mode i when 

there is no failure (B-7) 

P FC - probability of choosing Hj_ when Hj is true (B-8) 

In order to compute the above, several intermediate probabilities are 
needed. 

The probability of false trigger, P FT , is the probability that T is true 
when there is no failure. Using Eq. B-4, we have 

_ 7 

T » n Ti (B-9) 

i=l 


and 


P CR = P(T |Ho) = P(Ti|H 0 ) 7 = [l-P(Ti|H 0 )] 7 (B-10) 

assuming independent tests. Finally, P FT = 1 -Pqr. 

Similarly, the probability of false verification, P FV , is the probability 
that V is true under H 0 and is computed from 

P FV = P(V|H 0 = l-[l-P(Vi|H 0 )]7 ( B — 11) 

again assuming independent tests. 

In the following, the a priori probabilities of each failure mode are 
assumed to be equal, and all tests are assume to be independent. 

The false alarm probability, Pp^, is given by 

p FA - PI U di|H 0 ] ( B-12) 

i*0 
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Since the events, d^, are disjoint, 

PFA = l P(di|H 0 ) 
i 

= l Pft * P ( V i | H 0 ) ♦ n P(Ii/j|H 0 ) (B-13) 

i j = l 

Assuming that P(Vi|H 0 ) is the same for all i and Pdi/j|H 0 ) is the same for 
all j , we have 

PFA - 7 p FT * P(Vi|Ho • P(Ii/j|H 0 ) 6 (B-14) 

Assuming that choosing % over H j , i*j*o, when H 0 is true is completely 
random, i.e., P(Ii/j|H 0 ) = 1/2, we have, 

P FA ^ * P FT * P( v i|H 0 ) (B-15) 


The probability of false classification, P FC , is computed from 
PFC = 1-p CC where 


p CC = p 


U di n Hi 
i*0 


U 

i*0 



(B-16) 


Using Bayes rule and the fact that the events di fl Hi are disjoint, Eq. B-16 
can be rewritten, 

Pec = I P(dj|Hj) • P(Hj | u Hj) (B-17) 

j 

or when all of the a priori probabilities are equal, 

, 7 

P CC = 7 l PCdjlHj) ( B- 1 8 ) 

j=l 
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Now using Eq* B-6, the fact that each Vj_ and Ii/j event is a subset of T, 
each test is independent, and the assumption that the individual test proba- 
bilities are equal for each failure mode, we have 

P CC = PCTlHi) P(V ± | Hi ) . Pdi/jlH,)®' (B-19) 

< PCTilHi) P(V ± | Hi) Pdi/jlHi)^ (B-20) 

Example 

Suppose PdilHi) - PdilHo) = P(Vi|H 0 ) = P(Vi|Hi) - Pdj./j | H j ,j *i) = 10 -4 . 
Then we have, 

PpX ^ 10“ 3 
PFA < 10" 1 8 
Pp C < 10" 3 

The false alarm spec, if expressed as 1 in 10® samples, represents better than 
1 in 800 hours of flight time for a .03 second sample interval. 
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APPENDIX C 

FLIGHT DATA PROCESSING DETAILS 


The flight data values used by the Failure Detection and Isolation system 
are derived by converting the flight data of the tape into the required 
formats, by use of two external programs, CRUNCH and CONFIG. CRUNCH inputs 
the ASCII data and converts it into a binary, sequential file. CONFIG then 
converts the binary data into FDI required values and outputs them into 
another binary, sequential file. The conversions are as follows: 

1. converts velocity from knots to feet/sec 

2. computes the yaw rate by taking the mean of the two intermediate 
values from four sensor readings 

3. the left and right thrust are computed from throttle positions based 
on a first-order system 

a. first, the previous thrust value is initialized 

prev_lef t_J:hrust = .298 * lef t_throttle_j>os 
prev__right__thrust = .298 * right_throttle__pos 

b. then, for every time step: 

average_throttle = (lef t_throttle_j?os + right_throttle_pos)/2 
left__thrust = exp(-0.5 * delta_time)*prev_lef t-thrust + (exp(-0.5 

* delta_time)-l)*-2*.298*lef t__throttle 

prev_JLef t_thrust - left_thrust 

right-thrust = exp(-0.5*delta__time)*prev__right__thrust + (exp(-0.5 

* delta_time)-l)*-2) . 298*right_throttle 

prev_right_thrust = right_thrust 

4. invert normal acceleration measurement so that it is consistent with 
residual generation definition 
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5* compute stabilizer position from pilot units 
stab = 3 - pilot_unit 

6* correct rudder for bias in measurement 
rudder = rudder - 5*6 

7. make the aileron measurements complementary of each other 

left_aileron = - left_aileron 
right_aileron - right_aileron 

8. compute spoiler positions 

lef t__spoiler = (left_spoill + lef t__spoil2)/2 
right_spoiler * (right_8poi.il + right_jspoil2)/2 

All other input values are channeled to the output file untouched. In 

addition, this program initializes the variable QBAR (dynamic pressure) to 

zero so that it can be computed in FDI. The output file is headed by an 

integer indicating the number of channels in the output file. This is 

followed by the channels themselves, listed by vectors of time. The first 

nine measured surface deflections are repeated at the end to serve as dummy 

commanded deflection values. A list of the output file channels is given 

below: 

0 TIME 

1 V_T 

2 ALPHA 

3 BETA 

4 P 

5 Q 

6 R 

7 AX 

8 AY 

9 AZ 

10 QBAR 

1 1 ALTITUDE 
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measured values: 


12 

13 

14 

15 

16 

17 

18 

19 

20 
21 
22 


commanded values: 


23 

24 

25 

26 

27 

28 
29 


LEFT THRUST 
RIGHT THRUST 
STABILIZER 
STABILIZER 
RUDDER 

LEFT ELEVATOR 
RIGHT ELEVATOR 
LEFT AILERON 
RIGHT AILERON 
LEFT SPOILER 
RIGHT SPOILER 

LEFT THRUST 
RIGHT THRUST 
STABILIZER 
STABILIZER 
RUDDER 

LEFT ELEVATOR 
RIGHT ELEVATOR 
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